# Agentic Codebase Analysis & Issue Creation Pipeline

## Overview

This document describes a two-stage automated pipeline for continuous codebase improvement:

1. **Daily Codebase Analysis Pipeline** - Deep inspection of main branch with automated issue creation
2. **Multi-Agent Backlog Refinement Pipeline** - Scheduled review of backlog items by multiple agents

The workflow follows the existing pattern: Issues are created with analysis data, admin reviews and approves via `[Approved]` comment, then the Issue Monitor Agent creates PRs.

```
[Daily Analysis] → [Issues Created] → [Admin Review] → [Approved] → [PR Created]
                         ↓
              [Backlog Refinement]
                         ↓
           [Multi-Agent Comments]
```

---

## Part 1: Daily Codebase Analysis Pipeline

### Purpose

Automatically pull main branch, perform deep analysis, and create GitHub issues for:
- Code quality improvements
- Security vulnerabilities
- Performance optimizations
- Technical debt reduction
- Documentation gaps
- Test coverage improvements
- Dependency updates
- Architectural refactoring opportunities

### Architecture

```
┌─────────────────────────────────────────────────────────────────────┐
│                    Daily Codebase Analysis Pipeline                  │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐          │
│  │   Pull Main  │───▶│Deep Analysis │───▶│ Deduplicate  │          │
│  │    Branch    │    │   Agents     │    │  & Filter    │          │
│  └──────────────┘    └──────────────┘    └──────────────┘          │
│                             │                   │                    │
│                             ▼                   ▼                    │
│                      ┌──────────────┐    ┌──────────────┐          │
│                      │   Claude     │    │Create Issues │          │
│                      │   Gemini     │    │  On Board    │          │
│                      │   Codex      │    └──────────────┘          │
│                      │   OpenCode   │           │                   │
│                      └──────────────┘           ▼                   │
│                                          ┌──────────────┐          │
│                                          │Admin Review  │          │
│                                          │  Required    │          │
│                                          └──────────────┘          │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘
```

### Analysis Categories

Each agent specializes in different analysis types:

| Agent | Primary Focus | Analysis Types |
|-------|---------------|----------------|
| Claude | Architecture & Design | Patterns, refactoring, complexity |
| Gemini | Code Quality & Security | Vulnerabilities, code smells |
| Codex | Performance & Efficiency | Optimization opportunities |
| OpenCode | Tech Debt & Maintenance | Dependencies, deprecations |

### Issue Creation Format

Generated issues follow a standardized format for easy review:

```markdown
## [Analysis Type]: Brief Description

**Category**: Security / Performance / Quality / Tech Debt / Documentation
**Priority**: P0 (Critical) / P1 (High) / P2 (Medium) / P3 (Low)
**Effort Estimate**: XS / S / M / L / XL
**Discovered By**: claude / gemini / codex / opencode
**Analysis Run**: 2025-01-09

### Summary
[1-2 sentence description of the issue]

### Details
[Detailed explanation with code references]

### Affected Files
- `path/to/file1.py:L42-L58`
- `path/to/file2.py:L100`

### Suggested Fix
[Agent's recommended approach]

### Evidence
[Code snippets, metrics, or analysis output supporting the issue]

---
*Generated by Codebase Analysis Pipeline*
*Awaiting admin review - reply with `[Approved]` to create PR*
```

### Deduplication Strategy

Before creating new issues:

1. **Search existing issues** - Query open issues for similar titles/content
2. **Memory lookup** - Check if similar issue was recently created
3. **Semantic similarity** - Use embeddings to find conceptually similar issues
4. **Hash-based tracking** - Store hash of (file, line range, issue type) to prevent exact duplicates

### Pipeline Configuration

```yaml
# codebase-analysis-config.yml

analysis:
  # Schedule: Run daily at 3 AM UTC (off-peak hours)
  schedule: "0 3 * * *"

  # Analysis scope
  include_paths:
    - "packages/**/*.py"
    - "tools/**/*.py"
    - "automation/**/*.py"
  exclude_paths:
    - "**/tests/**"
    - "**/__pycache__/**"
    - "**/node_modules/**"

  # Issue limits (to prevent flooding)
  max_issues_per_run: 5
  max_issues_per_category: 2

  # Priority thresholds (only create issues above threshold)
  min_priority: P2  # Skip P3 (low priority) issues

  # Agent selection
  agents:
    - claude    # Architecture analysis
    - gemini    # Security analysis
    - codex     # Performance analysis

issue_creation:
  # Auto-add to project board
  add_to_board: true
  board_status: "Todo"

  # Labels to apply
  labels:
    - "automated"
    - "needs-review"
    - "agentic-analysis"

  # Assign to admin for review
  assignee: null  # null = unassigned, let admin self-assign

deduplication:
  # Check window for duplicates
  lookback_days: 30

  # Similarity threshold (0.0-1.0)
  similarity_threshold: 0.8

  # Store dedup hashes in memory service
  use_memory: true
```

---

## Part 2: Multi-Agent Backlog Refinement Pipeline

### Purpose

Periodically review backlog items with multiple agents to:
- Add unique insights as comments
- Identify blockers or dependencies
- Suggest implementation approaches
- Flag issues that may be obsolete
- Prioritize or reprioritize based on new context

### Architecture

```
┌─────────────────────────────────────────────────────────────────────┐
│                 Multi-Agent Backlog Refinement Pipeline             │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  ┌──────────────┐    ┌──────────────────────────────────┐          │
│  │Query Backlog │───▶│      Agent Review Round          │          │
│  │    Items     │    │                                   │          │
│  └──────────────┘    │  ┌─────────┐  ┌─────────┐        │          │
│                      │  │ Claude  │  │ Gemini  │        │          │
│                      │  └────┬────┘  └────┬────┘        │          │
│                      │       │            │              │          │
│                      │  ┌────▼────┐  ┌────▼────┐        │          │
│                      │  │ Insight │  │ Insight │        │          │
│                      │  └────┬────┘  └────┬────┘        │          │
│                      │       │            │              │          │
│                      │  ┌─────────┐  ┌─────────┐        │          │
│                      │  │ Codex   │  │OpenCode │        │          │
│                      │  └────┬────┘  └────┬────┘        │          │
│                      │       │            │              │          │
│                      │  ┌────▼────┐  ┌────▼────┐        │          │
│                      │  │ Insight │  │ Insight │        │          │
│                      │  └────┬────┘  └────┬────┘        │          │
│                      │       └──────┬──────┘            │          │
│                      └──────────────┼───────────────────┘          │
│                                     ▼                               │
│                      ┌──────────────────────────────────┐          │
│                      │     Deduplicate & Post           │          │
│                      │     Unique Comments              │          │
│                      └──────────────────────────────────┘          │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘
```

### Agent Review Prompts

Each agent reviews issues with specialized focus:

**Claude Code** (Architecture Focus):
```
Review this GitHub issue and provide architectural insights:
1. Are there design patterns that would help?
2. Are there existing utilities in the codebase that could be reused?
3. Are there potential breaking changes or migration needs?
4. What's the recommended implementation order?

Only comment if you have a UNIQUE insight not already in the issue or comments.
```

**Gemini CLI** (Quality Focus):
```
Review this GitHub issue for quality considerations:
1. Are there security implications?
2. Are there edge cases not mentioned?
3. What test scenarios should be covered?
4. Are there related issues that should be linked?

Only comment if you have a UNIQUE insight not already captured.
```

**Codex** (Implementation Focus):
```
Review this GitHub issue from an implementation perspective:
1. What's the estimated complexity?
2. Are there performance considerations?
3. What dependencies or blockers exist?
4. Can you suggest a concrete implementation approach?

Only comment if adding NEW information to the discussion.
```

**OpenCode** (Maintenance Focus):
```
Review this GitHub issue for maintainability:
1. Will this create technical debt?
2. Are there documentation requirements?
3. Does this need coordination with other systems?
4. Should this be broken into smaller issues?

Only add a comment if you have something UNIQUE to contribute.
```

### Comment Deduplication

Before posting a comment:

1. **Extract existing insights** - Parse all existing comments
2. **Semantic comparison** - Compare new insight against existing
3. **Keyword overlap** - Check for significant word overlap
4. **Agent attribution** - Track which agents have already commented

### Comment Format

```markdown
### Insight from [Agent Name]

[The unique insight or suggestion]

#### Context
[Supporting details or code references]

---
*Backlog refinement by [agent] on [date]*
*This is an automated analysis - human review recommended*
```

### Tracking Comments

Use HTML comment markers to track refinement:

```html
<!-- backlog-refinement:claude:2025-01-09:abc123 -->
```

This prevents the same agent from commenting twice on the same issue within a configurable window.

### Pipeline Configuration

```yaml
# backlog-refinement-config.yml

refinement:
  # Schedule: Run twice weekly (Tuesday and Friday)
  schedule: "0 10 * * 2,5"

  # Query parameters
  query:
    status: "Todo"           # Only review Todo items
    min_age_days: 3          # Skip recently created issues
    max_age_days: 90         # Skip very old issues
    exclude_labels:
      - "blocked"
      - "wontfix"
      - "in-progress"

  # Agent rotation (different agents each run)
  agents:
    round_1:  # First run of week
      - claude
      - gemini
    round_2:  # Second run of week
      - codex
      - opencode

  # Or use all agents each time
  # agents: [claude, gemini, codex, opencode]

  # Limits
  max_issues_per_run: 10
  max_comments_per_issue: 2

comment_rules:
  # Minimum insight length
  min_length: 50

  # Maximum insight length
  max_length: 500

  # Similarity threshold for dedup
  similarity_threshold: 0.7

  # Cooldown: Don't let same agent comment on same issue within X days
  agent_cooldown_days: 14
```

---

## Implementation Plan

### Phase 1: Core Infrastructure

1. **Create analysis agent base class**
   - `packages/github_agents/src/github_agents/analyzers/base.py`
   - Abstract class for codebase analysis
   - File reading, pattern matching, metric collection

2. **Implement issue creator**
   - `packages/github_agents/src/github_agents/creators/issue_creator.py`
   - Create issues from analysis results
   - Add to project board
   - Track in memory for deduplication

3. **Create deduplication service**
   - Use existing memory integration
   - Store issue fingerprints
   - Semantic similarity checking

### Phase 2: Analysis Agents

1. **Claude Architecture Analyzer**
   - Complexity analysis
   - Pattern detection
   - Refactoring opportunities

2. **Gemini Security Analyzer**
   - Vulnerability scanning
   - Code smell detection
   - Best practice violations

3. **Codex Performance Analyzer**
   - Bottleneck detection
   - Optimization opportunities
   - Resource usage analysis

4. **OpenCode Maintenance Analyzer**
   - Dependency analysis
   - Documentation gaps
   - Tech debt identification

### Phase 3: Backlog Refinement

1. **Refinement monitor**
   - `packages/github_agents/src/github_agents/monitors/refinement.py`
   - Query backlog items
   - Orchestrate agent reviews
   - Post deduplicated comments

2. **Comment tracker**
   - Parse existing comments
   - Track agent contributions
   - Prevent duplicate insights

### Phase 4: GitHub Workflows

1. **Daily analysis workflow**
   - `.github/workflows/codebase-analysis.yml`
   - Schedule: Daily at 3 AM UTC
   - Uses board-agent-worker pattern

2. **Backlog refinement workflow**
   - `.github/workflows/backlog-refinement.yml`
   - Schedule: Twice weekly
   - Parallel agent execution

### Phase 5: Configuration & Documentation

1. **Config files**
   - `codebase-analysis-config.yml`
   - `backlog-refinement-config.yml`

2. **Documentation**
   - Update CLAUDE.md
   - Create operation guide
   - Add troubleshooting section

---

## File Structure

```
packages/github_agents/
├── src/github_agents/
│   ├── analyzers/
│   │   ├── __init__.py
│   │   ├── base.py              # Base analyzer class
│   │   ├── architecture.py      # Claude: architecture analysis
│   │   ├── security.py          # Gemini: security analysis
│   │   ├── performance.py       # Codex: performance analysis
│   │   └── maintenance.py       # OpenCode: maintenance analysis
│   │
│   ├── creators/
│   │   ├── __init__.py
│   │   └── issue_creator.py     # Issue creation with dedup
│   │
│   ├── monitors/
│   │   ├── issue.py             # (existing)
│   │   ├── pr.py                # (existing)
│   │   └── refinement.py        # NEW: Backlog refinement
│   │
│   └── cli.py                   # Add new commands:
│                                # - analyze-codebase
│                                # - refine-backlog

.github/workflows/
├── codebase-analysis.yml        # Daily analysis pipeline
└── backlog-refinement.yml       # Bi-weekly refinement pipeline

config/
├── codebase-analysis-config.yml
└── backlog-refinement-config.yml
```

---

## Security Considerations

1. **Rate Limiting** - Prevent API exhaustion
   - Max issues created per day
   - Max comments per issue
   - Cooldown between runs

2. **Access Control** - Issues still require admin approval
   - All generated issues marked `needs-review`
   - `[Approved]` keyword still required for PR creation
   - Agents cannot self-approve

3. **Content Validation** - Prevent injection
   - Sanitize agent outputs
   - Validate issue content
   - Filter sensitive information

4. **Audit Trail** - Track all automated actions
   - Log all issue creations
   - Track comment history
   - Store in memory for analysis

---

## Metrics & Monitoring

Track pipeline effectiveness:

1. **Creation Metrics**
   - Issues created per run
   - Issues approved vs rejected
   - Time to approval
   - PR success rate

2. **Refinement Metrics**
   - Comments added per run
   - Unique insights vs duplicates
   - Agent contribution distribution

3. **Quality Metrics**
   - Issues that led to successful PRs
   - Agent accuracy by category
   - False positive rate

---

## Example Workflow Output

### Daily Analysis Run

```
=== Codebase Analysis Pipeline ===
Run: 2025-01-09 03:00:00 UTC

Agents Used:
- Claude (architecture)
- Gemini (security)
- Codex (performance)

Analysis Results:
- Claude: 3 findings
- Gemini: 2 findings
- Codex: 1 finding

Deduplication:
- 2 duplicates filtered (similar to existing issues)
- 1 below priority threshold (P3)

Issues Created: 3
- #142: [Security] SQL injection risk in query builder (P1)
- #143: [Performance] N+1 query in user loader (P2)
- #144: [Architecture] Circular dependency in utils (P2)

All issues added to board with status: Todo
Awaiting admin review.
```

### Backlog Refinement Run

```
=== Backlog Refinement Pipeline ===
Run: 2025-01-09 10:00:00 UTC

Issues Reviewed: 8
Agents: Claude, Gemini

Comments Added:
- Issue #98: Claude added implementation suggestion
- Issue #102: Gemini identified security consideration
- Issue #105: (skipped - all insights already present)
- Issue #108: Claude suggested breaking into smaller issues

Summary:
- 3 unique insights added
- 5 issues had no new insights to add
- 0 issues flagged for closure
```

---

## Next Steps

1. Review and approve this design
2. Implement Phase 1 (core infrastructure)
3. Add first analyzer (Claude architecture)
4. Create daily workflow
5. Test with dry-run mode
6. Enable scheduled runs
7. Implement refinement pipeline
8. Monitor and tune parameters
