# APE Framework Implementation Strategy

## Overview

This document outlines the detailed implementation strategy for the Automatic Prompt Engineer (APE) framework, including prompt candidate generation algorithms, evaluation mechanisms, and optimization workflows.

## Implementation Architecture

### Core Components Implementation

#### 1. Prompt Candidate Generator

**Purpose**: Generate diverse, high-quality prompt candidates using multiple strategies

**Implementation Approach**:

```typescript
// Pseudo-implementation structure
class PromptCandidateGenerator {
  async generateCandidates(
    basePrompt: PromptObject,
    strategies: GenerationStrategy[],
    count: number
  ): Promise<PromptCandidate[]>;
}
```

**Generation Strategies**:

##### Template-based Variation

- **Strategy**: Use predefined templates with variable substitution
- **Templates**:
  - Imperative: "Please `{action}` the `{subject}` by `{method}`..."
  - Collaborative: "Let's work together to `{action}` `{subject}`..."
  - Instructional: "To `{action}` `{subject}`, follow these steps..."
  - Question-based: "How can we `{action}` `{subject}` using `{method}`?"

##### Semantic Variation

- **Synonym Replacement**: Replace key terms with domain-appropriate synonyms
- **Phrase Restructuring**: Reorganize sentence structure while preserving meaning
- **Perspective Shifting**: Change from different viewpoints (user, system, expert)
- **Abstraction Level**: Adjust between high-level and detailed instructions

##### Style Variation

- **Formal Style**: "Please conduct a comprehensive analysis..."
- **Conversational Style**: "Let's take a look at this project and see..."
- **Technical Style**: "Execute architectural analysis using established patterns..."
- **Instructional Style**: "Step 1: Analyze the codebase. Step 2: Identify patterns..."

#### 2. Evaluation Engine

**Purpose**: Score prompt candidates using multiple evaluation criteria

**Evaluation Criteria Implementation**:

##### Task Completion (Weight: 30%)

- **Metric**: How well the prompt achieves the intended task
- **Evaluation**: Compare expected vs actual outcomes
- **Scoring**: 0-1 scale based on task success rate

##### Clarity (Weight: 25%)

- **Metric**: How clear and unambiguous the prompt is
- **Evaluation**: Analyze sentence structure, word choice, organization
- **Scoring**: Readability scores, ambiguity detection

##### Specificity (Weight: 20%)

- **Metric**: How specific and actionable the prompt is
- **Evaluation**: Count specific instructions, concrete examples
- **Scoring**: Ratio of specific to general statements

##### Robustness (Weight: 15%)

- **Metric**: How well the prompt handles edge cases
- **Evaluation**: Test with various input scenarios
- **Scoring**: Success rate across different contexts

##### Efficiency (Weight: 10%)

- **Metric**: How concise yet comprehensive the prompt is
- **Evaluation**: Information density, redundancy analysis
- **Scoring**: Information per token ratio

#### 3. Selection Algorithm

**Purpose**: Choose optimal prompts from evaluated candidates

**Selection Strategies**:

##### Multi-criteria Selection (Recommended)

```typescript
function selectOptimalPrompts(
  candidates: EvaluationResult[],
  criteria: EvaluationCriterion[],
  weights: Record<EvaluationCriterion, number>
): PromptCandidate[];
```

**Algorithm**:

1. **Weighted Scoring**: Calculate weighted average of all criteria
2. **Pareto Optimization**: Find candidates that are not dominated by others
3. **Diversity Filtering**: Ensure selected prompts are sufficiently different
4. **Quality Threshold**: Filter candidates below minimum quality threshold

## Optimization Workflow

### Phase 1: Initial Candidate Generation

**Duration**: 30-60 seconds
**Process**:

1. **Strategy Selection**: Choose appropriate generation strategies based on task type
2. **Parallel Generation**: Generate candidates using multiple strategies simultaneously
3. **Quality Filtering**: Remove obviously poor candidates early
4. **Diversity Checking**: Ensure sufficient diversity in candidate pool

### Phase 2: Comprehensive Evaluation

**Duration**: 60-120 seconds
**Process**:

1. **Automated Evaluation**: Apply all evaluation criteria to each candidate
2. **Context Matching**: Assess how well candidates fit the specific context
3. **Performance Prediction**: Estimate likely performance of each candidate
4. **Bias Detection**: Check for potential biases in prompts

### Phase 3: Selection and Refinement

**Duration**: 30-60 seconds
**Process**:

1. **Multi-criteria Selection**: Select top candidates using weighted criteria
2. **Ensemble Creation**: Combine strengths of multiple good candidates
3. **Refinement Generation**: Create refined versions of top candidates
4. **Final Validation**: Validate selected prompts against requirements

### Phase 4: Optimization Loop (Optional)

**Duration**: Variable (1-3 iterations)
**Process**:

1. **Performance Feedback**: Collect feedback from initial prompt usage
2. **Pattern Analysis**: Identify successful patterns in top-performing prompts
3. **Targeted Generation**: Generate new candidates based on successful patterns
4. **Convergence Check**: Determine if further optimization is beneficial

## Integration with MCP Tools

### High-Priority Tool Optimizations

#### 1. ADR Generation Tools

**Target Tools**: `generate_adrs_from_prd`, `suggest_adrs`
**Optimization Focus**:

- **Context Analysis**: Better understanding of PRD requirements
- **ADR Structure**: Optimal ADR format and content organization
- **Decision Rationale**: Clearer explanation of architectural decisions

**APE Configuration**:

```typescript
const adrOptimizationConfig: APEConfig = {
  candidateCount: 7,
  evaluationCriteria: ['task-completion', 'specificity', 'clarity', 'robustness'],
  optimizationRounds: 3,
  selectionStrategy: 'multi-criteria',
  cacheEnabled: true,
  performanceTracking: true,
  maxOptimizationTime: 180000, // 3 minutes
  qualityThreshold: 0.7,
  diversityWeight: 0.3,
};
```

#### 2. Analysis Tools

**Target Tools**: `analyze_project_ecosystem`, `get_architectural_context`
**Optimization Focus**:

- **Technology Detection**: Better identification of technologies and patterns
- **Context Extraction**: More comprehensive context analysis
- **Insight Generation**: Deeper architectural insights

#### 3. Research Tools

**Target Tools**: `generate_research_questions`, `incorporate_research`
**Optimization Focus**:

- **Question Quality**: More targeted and valuable research questions
- **Research Integration**: Better integration of research findings
- **Knowledge Synthesis**: Improved synthesis of multiple research sources

### Tool Integration Pattern

```typescript
// Example: APE-enhanced tool implementation
export async function generateOptimizedPrompt(
  toolName: string,
  basePrompt: PromptObject,
  context: any,
  config?: Partial<APEConfig>
): Promise<{ prompt: string; instructions: string; context: any }> {
  // Step 1: Get tool-specific APE configuration
  const apeConfig = getToolAPEConfig(toolName, config);

  // Step 2: Generate optimization prompt for AI delegation
  const optimizationPrompt = `
# Automatic Prompt Engineering Request

Please optimize the following prompt using APE techniques for the ${toolName} tool.

## Original Prompt
\`\`\`
${basePrompt.prompt}
\`\`\`

## Original Instructions
\`\`\`
${basePrompt.instructions}
\`\`\`

## Context
${JSON.stringify(context, null, 2)}

## APE Configuration
- **Candidate Count**: ${apeConfig.candidateCount}
- **Evaluation Criteria**: ${apeConfig.evaluationCriteria.join(', ')}
- **Selection Strategy**: ${apeConfig.selectionStrategy}
- **Quality Threshold**: ${apeConfig.qualityThreshold}

## Optimization Tasks

### Step 1: Generate Prompt Candidates
Create ${apeConfig.candidateCount} prompt variations using these strategies:
1. **Template Variation**: Use different prompt templates
2. **Semantic Variation**: Rephrase while preserving meaning
3. **Style Variation**: Adjust tone and style
4. **Structure Variation**: Reorganize prompt structure
5. **Specificity Variation**: Adjust detail level

### Step 2: Evaluate Candidates
Evaluate each candidate on:
${apeConfig.evaluationCriteria.map(criterion => `- **${criterion}**: Score 0-1 based on ${getEvaluationDescription(criterion)}`).join('\n')}

### Step 3: Select Optimal Prompt
Use ${apeConfig.selectionStrategy} strategy to select the best prompt considering:
- Weighted evaluation scores
- Diversity requirements
- Quality threshold (${apeConfig.qualityThreshold})
- Context appropriateness

## Expected Output Format
\`\`\`json
{
  "optimizedPrompt": {
    "prompt": "optimized prompt text",
    "instructions": "optimized instructions",
    "context": { "optimization_metadata": "..." }
  },
  "optimization": {
    "candidatesGenerated": number,
    "candidatesEvaluated": number,
    "improvementScore": number,
    "optimizationReasoning": "explanation of improvements",
    "evaluationScores": {
      "task-completion": number,
      "clarity": number,
      "specificity": number
    }
  }
}
\`\`\`

## Quality Requirements
- Optimized prompt must score above ${apeConfig.qualityThreshold} on all criteria
- Must maintain original task objectives
- Should improve clarity and effectiveness
- Must be appropriate for the ${toolName} context
`;

  const instructions = `
# APE Optimization Instructions

You must:
1. **Generate Diverse Candidates**: Create varied prompt alternatives using multiple strategies
2. **Evaluate Systematically**: Score each candidate on all specified criteria
3. **Select Optimally**: Choose the best prompt using the specified selection strategy
4. **Validate Quality**: Ensure the optimized prompt meets quality thresholds
5. **Document Improvements**: Explain how the optimized prompt is better

## Success Criteria
- Optimized prompt scores higher than original on evaluation criteria
- Maintains task objectives and context appropriateness
- Provides clear improvement reasoning
- Follows exact JSON output format
`;

  return {
    prompt: optimizationPrompt,
    instructions,
    context: {
      operation: 'ape_optimization',
      toolName,
      originalPrompt: basePrompt,
      apeConfig,
      securityLevel: 'high',
      expectedFormat: 'json',
    },
  };
}
```

## Performance Optimization

### Caching Strategy

1. **Candidate Cache**: Cache generated candidates by strategy and context
2. **Evaluation Cache**: Cache evaluation results for reuse
3. **Optimization Cache**: Cache final optimized prompts
4. **Performance Cache**: Cache performance metrics and feedback

### Resource Management

- **Parallel Processing**: Generate and evaluate candidates in parallel
- **Memory Limits**: Implement memory usage limits for large optimization tasks
- **Time Limits**: Set maximum optimization time to prevent runaway processes
- **Quality Gates**: Stop optimization early if quality threshold is met

### Monitoring and Metrics

- **Optimization Success Rate**: Track percentage of successful optimizations
- **Performance Improvement**: Measure average improvement scores
- **Resource Usage**: Monitor CPU, memory, and time usage
- **User Satisfaction**: Collect feedback on optimized prompts

This implementation strategy provides a comprehensive roadmap for building the APE framework while maintaining the 100% prompt-driven architecture and ensuring optimal performance across MCP tools.
