# Performance Benchmarks

This directory contains performance baseline data and historical measurements for the tool installation setup process.

## Directory Structure

```
benchmarks/
├── README.md           # This file
├── baseline.json       # Current performance baseline (to be created by collect-baseline.sh)
├── budgets.json        # Performance budgets and thresholds (to be created)
└── history/            # Historical performance data
    └── {commit-sha}.json  # Performance measurements per commit
```

## Files

### baseline.json

Contains the reference performance measurements used for regression detection. This file is created by running `scripts/collect-baseline.sh` and should be updated after verified performance improvements or major refactoring.

**Schema:**

```json
{
  "baseline_version": "1.0",
  "collected_at": "ISO 8601 timestamp",
  "commit_sha": "git commit hash",
  "platform": "linux|macos|windows",
  "cold_cache": {
    "total_time_seconds": number,
    "categories": {...}
  },
  "warm_cache": {
    "total_time_seconds": number,
    "categories": {...}
  },
  "cache_effectiveness": {
    "speedup_percentage": number,
    "poorly_cached_tools": [...]
  }
}
```

### budgets.json

Defines performance budgets and regression thresholds for different tool categories. This file will be created as part of Task 10.1.

**Schema:**

```json
{
  "version": "1.0",
  "total_time_seconds": number,
  "categories": {
    "security": number,
    "linters": number,
    "formatters": number,
    "runtimes": number
  },
  "per_tool_budgets": {
    "tool-name": number
  },
  "thresholds": {
    "warning_percent": 20,
    "error_percent": 50
  }
}
```

### history/

Contains historical performance measurements organized by git commit SHA. These files are automatically generated by CI workflows and the performance test script.

**Filename format:** `{commit-sha}.json`

**Schema:** Same as the output from `scripts/test-performance.sh --output-format json`

## Usage

### Run Performance Test

```bash
# Run full performance test
./scripts/test-performance.sh

# Run with JSON output
./scripts/test-performance.sh --output-format json

# Test specific category
./scripts/test-performance.sh --categories security

# Dry run (preview without installation)
./scripts/test-performance.sh --dry-run
```

### Collect Baseline

```bash
# Collect baseline measurements (both cold and warm cache)
./scripts/collect-baseline.sh

# Collect only cold cache baseline
./scripts/collect-baseline.sh --cache-mode cold

# Collect only warm cache baseline
./scripts/collect-baseline.sh --cache-mode warm

# Save to custom location
./scripts/collect-baseline.sh --output /path/to/baseline.json
```

### Compare Performance

```bash
# Compare current performance against baseline (to be implemented in Task 1.3)
./scripts/compare-performance.sh
```

## Requirements

- **Requirement 1.1:** Performance baseline establishment
- **Requirement 1.2:** Individual tool category measurements
- **Requirement 1.3:** Structured data format (JSON)
- **Requirement 1.4:** CI and local environment execution
- **Requirement 1.5:** System metadata recording

## Maintenance

- Update `baseline.json` after verified performance improvements
- Review performance budgets quarterly
- Clean up old historical data (keep last 100 measurements)
- Archive older measurements to separate storage if needed

## Related Documentation

- [Design Document](../.kiro/specs/performance-testing-and-docs/design.md)
- [Requirements](../.kiro/specs/performance-testing-and-docs/requirements.md)
- [Tasks](../.kiro/specs/performance-testing-and-docs/tasks.md)
