# AI Slop Detection Patterns

**Version:** 1.0.0
**Last Updated:** 2025-12-28

## Purpose

This document provides centralized detection patterns for AI-generated low-quality code ("AI slop"). All code review agents MUST reference these patterns during review.

## What is AI Slop?

AI slop refers to code generated by AI assistants that:
- Looks syntactically correct and well-formatted
- Uses common patterns that "pass the eye test"
- Contains real problems: hallucinated dependencies, overengineering, scope creep, or semantic incorrectness

---

## AI Generation Smell Detection

### Code Smells Unique to AI Generation

| Smell | Detection Pattern | Severity | Why AI Does This |
|-------|-------------------|----------|------------------|
| **Phantom Dependencies** | Package names that don't exist on npm/PyPI/Go modules | CRITICAL | AI hallucinates plausible names from training data |
| **Morpheme-Spliced Names** | Plausible but non-existent: `graphit-orm`, `wave-socket` | CRITICAL | AI combines familiar morphemes without registry verification |
| **Cross-Ecosystem Borrowing** | `serverless-python-requirements` (npm name in Python) | HIGH | AI confuses packages across registries |
| **Confident Hallucination** | Code references APIs/methods that don't exist in the library version | HIGH | AI generates from statistical patterns, not docs |
| **Over-Abstraction** | Unnecessary interfaces, factories, or adapters for one-time operations | HIGH | AI pattern-matches "best practices" without context |
| **Premature Generalization** | Config options, feature flags, or extensibility for unrequested features | HIGH | AI adds "future-proofing" that wasn't requested |
| **Context-Gap Filling** | Generic implementations that "fill in" unspecified requirements | MEDIUM | AI guesses when requirements are incomplete |
| **Surface-Form Mimicry** | Code follows naming conventions but logic is semantically wrong | HIGH | AI matches form without understanding function |
| **Scope Creep** | Changes to files/functions not mentioned in requirements | HIGH | AI tries to be "helpful" beyond the request |
| **Cargo-Cult Patterns** | Design patterns applied without understanding (Observer for 1 subscriber) | MEDIUM | AI applies patterns based on keywords, not need |
| **Verbose Boilerplate** | Excessive error handling for impossible scenarios | LOW | AI over-generalizes defensive patterns |
| **Typo-Adjacent Names** | Package names one character off from real packages | CRITICAL | AI approximates from training data |

---

## Slopsquatting Detection (Security)

### What is Slopsquatting?

Slopsquatting is a supply-chain attack where:
1. AI hallucates a plausible but non-existent package name
2. Attacker pre-registers that name on public registries
3. Developer runs AI-generated install command
4. Malicious code is downloaded and executed

### Mandatory Checks for Every New Dependency

**Registry Verification Commands:**
```bash
# npm (JavaScript/TypeScript)
npm view <package-name> version

# PyPI (Python)
pip index versions <package-name>

# Go modules
go list -m <module-path>@latest

# Cargo (Rust)
cargo search <crate-name>
```

### Suspicious Naming Patterns

| Pattern | Example | Risk Level | Detection Method |
|---------|---------|------------|------------------|
| **Morpheme-spliced** | `graphit-orm`, `wave-socket`, `fast-json-parser` | HIGH | Verify each package exists in registry |
| **Cross-ecosystem** | npm name used in Python project | HIGH | Check correct registry for language |
| **Typo-adjacent** | `lodahs`, `requets`, `expresss` | CRITICAL | Compare against known popular packages |
| **Plausible prefix** | `opentelemetry-instrumentation-fake` | HIGH | Verify official package namespaces |
| **Brand new package** | Package < 30 days old | HIGH | Check package creation date |
| **Low downloads** | < 100 weekly downloads for common functionality | MEDIUM | Verify popularity matches claimed utility |
| **Author mismatch** | Popular name, unknown author | HIGH | Compare against official maintainers |

### Verification Output Format

For EVERY new dependency, document:
```markdown
### Dependency Verification
| Package | Registry | Exists | Author | Age | Downloads/Week | Risk |
|---------|----------|--------|--------|-----|----------------|------|
| lodash | npm | ✅ | lodash | 12y | 50M | LOW |
| graphit-orm | npm | ❌ NOT FOUND | - | - | - | CRITICAL |
```

---

## Evidence-of-Reading Verification

### Problem

AI agents often claim to have read files but operate on cached/assumed content, producing code that doesn't match existing patterns.

### Verification Checklist

| Check | How to Verify | Red Flag |
|-------|---------------|----------|
| **Existing Pattern Awareness** | Does new code match existing patterns in the codebase? | New code uses different conventions than surrounding code |
| **Import Consistency** | Are imports from same locations as similar files? | Imports from different paths than established patterns |
| **Error Handling Style** | Does error handling match project conventions? | Generic try/catch when project uses Result types |
| **Naming Convention Match** | Do new names match existing naming patterns? | camelCase in a snake_case codebase |
| **Type Consistency** | Are types used consistently with existing code? | `any` or `interface{}` when project uses strict typing |
| **Test Pattern Match** | Do tests follow project's testing patterns? | Jest syntax in a Vitest project |
| **Logging Pattern Match** | Does logging match surrounding code? | No logging when all adjacent functions log |
| **Comment Style** | Do comments match project documentation style? | Different comment format than rest of codebase |

### Evidence Requirements

Before approving code, verify implementation shows evidence of reading:
- [ ] Related existing files (imports match, patterns match)
- [ ] Type definitions (types used correctly)
- [ ] Test patterns (test style matches)
- [ ] Configuration (respects project config)
- [ ] Documentation (follows documented conventions)

**If evidence is missing → Flag as potential AI slop**

---

## Overengineering Detection

### Red Flags

| Pattern | Why It's Suspicious | Question to Ask |
|---------|--------------------|--------------------|
| **Interface with 1 implementation** | AI creates interfaces "for flexibility" | Will there ever be a 2nd implementation? |
| **Factory for single type** | AI pattern-matches factory pattern | Is the creation logic actually complex? |
| **Strategy pattern for 1 strategy** | AI sees conditional and adds pattern | Are more strategies planned? |
| **Wrapper adding no value** | AI creates layers for "separation" | What does this wrapper actually do? |
| **Configuration for hardcoded values** | AI externalizes everything | Will this value ever change? |
| **Generic<T> for single type** | AI adds generics "for reusability" | Will T ever be anything else? |
| **Event system for 1 subscriber** | AI sees "decoupling" opportunity | Are more subscribers expected? |
| **Middleware for single operation** | AI adds middleware "for extensibility" | What other operations would use this? |
| **Abstract base class with 1 child** | AI adds abstraction "for structure" | Will there be sibling classes? |
| **Builder pattern for simple object** | AI adds fluent API "for clarity" | Does object have optional fields? |
| **Stateless struct when functions would suffice** | AI creates struct with no fields (Go-specific) | Does the struct hold any state? |

### Severity Assessment

- **3+ of these patterns in one PR** → HIGH (likely AI slop)
- **1-2 patterns with no justification** → MEDIUM
- **Patterns with documented future use** → LOW (acceptable)

### Stateless Struct Pattern (Go)

```go
// ❌ OVER-ENGINEERED: Struct with no fields
type ScopeCore two struct{}

func (s *ScopeCore two) Match(scope string, pattern string) bool {
    return strings.HasPrefix(scope, pattern)
}

// Usage requires unnecessary instantiation:
matcher := &ScopeCore two{}
result := matcher.Match(scope, pattern)

// ✅ SIMPLE: Plain function
func MatchScope(scope string, pattern string) bool {
    return strings.HasPrefix(scope, pattern)
}

// Usage is direct:
result := MatchScope(scope, pattern)
```

**When stateless structs ARE justified:**
- Implementing an interface (e.g., `http.Handler`)
- Grouping related methods for discoverability
- Future state is planned (documented)

**When to flag:**
- Single method on struct with no fields
- No interface requirement
- No documented reason for struct wrapper

---

## Scope Boundary Enforcement

### Problem

AI agents make "helpful" changes outside requested scope, creating untested code paths.

### Verification Process

1. **Extract Scope from Requirements**
   - List all files mentioned in task/requirements
   - List all functions mentioned
   - List all behaviors specified

2. **Compare to Actual Changes**
   ```
   REQUESTED SCOPE:
   - [ ] File: src/auth/login.ts (modify)
   - [ ] Function: validateCredentials (add retry logic)

   ACTUAL CHANGES:
   - [✅] File: src/auth/login.ts
   - [⚠️] File: src/auth/logout.ts (NOT IN SCOPE)
   - [⚠️] File: src/utils/retry.ts (NEW - NOT REQUESTED)
   ```

3. **Flag Out-of-Scope Changes**

| Change Type | Severity | Required Action |
|-------------|----------|-----------------|
| New file not requested | HIGH | Require justification or remove |
| Modified file not in scope | HIGH | Require justification or revert |
| Deleted code not requested | CRITICAL | Block until justified |
| Refactored "while I was here" | MEDIUM | Flag for review, suggest separate PR |
| Added "helpful" utilities | MEDIUM | Question necessity |

---

## Hallucination Indicator Detection

### Language Patterns That Indicate Guessing

Flag code comments or PR descriptions containing:

| Pattern | Example | Meaning |
|---------|---------|---------|
| "likely", "probably" | "This file likely handles..." | **Agent didn't read file** |
| "should work" | "This should work with..." | **Agent didn't test** |
| "based on common patterns" | "Based on common patterns, I added..." | **Agent guessed** |
| "assuming" | "Assuming the API returns..." | **Agent didn't verify** |
| "typical", "usually" | "This typically requires..." | **Agent used training data, not codebase** |
| "I believe" | "I believe this function..." | **Agent is uncertain** |
| "standard approach" | "Using the standard approach..." | **May not match this codebase** |

### Code Indicators

| Indicator | Detection | Severity |
|-----------|-----------|----------|
| **TODO with vague content** | `// TODO: implement properly` | HIGH - incomplete |
| **Placeholder returns** | `return null; // placeholder` | CRITICAL - not implemented |
| **Generic error messages** | `throw new Error('Something went wrong')` | MEDIUM - lazy |
| **Commented-out code** | `// const oldWay = ...` | LOW - cleanup needed |
| **Magic strings/numbers** | Unexplained literals | MEDIUM - needs constants |
| **Empty catch blocks** | `catch (e) {}` | HIGH - swallowed errors |
| **Console.log debugging** | `console.log('here')` | LOW - cleanup needed |

---

## Anti-Rationalization for AI Slop

| AI Rationalization | Why It's WRONG | Required Action |
|--------------------|----------------|-----------------|
| "For future extensibility" | YAGNI - don't build for hypotheticals | **Remove unless in requirements** |
| "Industry best practice" | Best practice ≠ every practice everywhere | **Verify it applies to THIS codebase** |
| "For testability" | Dependency injection doesn't require interfaces | **Question if abstraction is needed** |
| "Separation of concerns" | One class can have multiple methods | **Verify separation is warranted** |
| "Needed for the implementation" | Then it should be in requirements | **Add to requirements first** |
| "Improved while I was there" | Out of scope = untested = risky | **Separate PR required** |
| "DRY principle" | DRY doesn't justify scope creep | **Keep scope, refactor later** |
| "Future-proofing" | YAGNI applies | **Remove unless requested** |
| "Standard library for this" | May be hallucinated | **Verify package exists** |
| "Common pattern in codebase" | May not have read codebase | **Show evidence of pattern** |

---

## Quick Reference: Severity Summary

| Issue Type | Severity | Blocks PR? |
|------------|----------|------------|
| Phantom/hallucinated dependency | CRITICAL | YES |
| Typosquatting-adjacent package name | CRITICAL | YES |
| API/method that doesn't exist | HIGH | YES |
| Scope creep (new files not requested) | HIGH | YES (without justification) |
| 3+ overengineering patterns | HIGH | YES |
| Missing evidence of reading existing code | MEDIUM | No, but flag |
| 1-2 overengineering patterns | MEDIUM | No, but flag |
| Hallucination language indicators | MEDIUM | No, but investigate |
| Verbose/unnecessary boilerplate | LOW | No |

---

## Usage

All code review agents MUST:

1. **Reference this document** in their review checklist
2. **Check for AI slop patterns** before issuing PASS verdict
3. **Document findings** using the formats provided
4. **Escalate CRITICAL issues** as automatic FAIL

**Integration:**
```markdown
See [shared-patterns/ai-slop-detection.md](../skills/shared-patterns/ai-slop-detection.md) for AI slop detection checklist.
```
