# mcp-perturbation Testing Status

## ✅ RESOLVED: Successfully Migrated to GEARS (January 2026)

The mcp-perturbation server has been successfully upgraded from scgen (2019) to GEARS (2024) with full Python 3.11 compatibility.

### Current Status

**Installation**: ✅ Working perfectly
**Python Version**: 3.11+ fully supported
**Dependencies**: All modern, no conflicts
**Performance**: 40% better than scgen (Nature Biotechnology 2024)

### Quick Installation

```bash
cd servers/mcp-perturbation
pip install -e .
```

**Installed packages:**
- cell-gears==0.1.2
- torch-geometric==2.7.0
- scanpy==1.11.5
- anndata==0.12.7
- torch==2.9.1
- All dependencies compatible!

### What Changed

| Aspect | scgen (old) | GEARS (new) |
|--------|-------------|-------------|
| Python | ❌ 3.9 only | ✅ 3.11+ |
| Architecture | VAE | Graph Neural Network |
| Performance | Baseline | +40% precision |
| Multi-gene | Limited | ✅ Excellent |
| Maintenance | Unmaintained (2021) | Active (2024) |
| Dependencies | Conflicts | ✅ Modern & compatible |

See [ALTERNATIVES_COMPARISON.md](ALTERNATIVES_COMPARISON.md) for detailed analysis.

---

## Previous Installation Status (scgen - ARCHIVED)

### Installation Status: ❌ Failed (scgen incompatible)

### ✅ Successfully Installed
- Python 3.11.13
- PyTorch 2.9.1
- scvi-tools 1.4.1
- scanpy 1.11.5
- All other core dependencies

### ❌ Known Issue: scGen Dependency Conflict

**Problem**: The standalone `scgen` package (v2.1.0) is incompatible with modern `scvi-tools` (v1.4.1+).

**Error**:
```
ModuleNotFoundError: No module named 'scvi._compat'
```

**Root Cause**: 
- scgen 2.1.0 was released in 2021
- scvi-tools has since evolved and removed the `_compat` module
- scGen is NO LONGER integrated into scvi-tools as a built-in model

## Recommended Solutions

### Option 1: Use Compatible Version - ❌ NOT VIABLE

**Attempted but failed due to dependency conflicts:**

```toml
dependencies = [
    "scgen==2.1.0",
    "scvi-tools>=0.15.0",  # scgen requires >=0.15.0, not 0.14.x
    "anndata<0.8",
    "scanpy>=1.7,<1.9",
]
```

**Issues discovered:**
1. scgen 2.1.0 requires scvi-tools>=0.15.0 (not 0.14.x)
2. scvi-tools 0.15-0.16 brings many JAX/PyTorch Lightning dependencies
3. Modern zarr (from anndata) requires numpy 2.0+
4. Old pandas (<1.5) has binary incompatibility with numpy 2.0+
5. numba requires numpy <2.4
6. Circular dependency conflicts in Python 3.11 environment

**Conclusion:** The 2021-era scgen package ecosystem is incompatible with Python 3.11 and modern package infrastructure.

### Option 2: Docker Container with Python 3.9 (QUICKEST FIX)

Create a Docker container with Python 3.9 and exact pinned versions:

```dockerfile
FROM python:3.9-slim
RUN pip install scgen==2.1.0 scvi-tools==0.16.4 numpy==1.23.5 pandas==1.4.4
```

This isolates the old package ecosystem from the modern Python 3.11 environment.

### Option 3: Implement Custom scGen (RECOMMENDED FOR PRODUCTION)

Fork scgen and modernize it for scvi-tools 1.x:
- https://github.com/theislab/scgen
- Update to use modern scvi-tools VAE API
- Remove deprecated _compat dependencies
- Update for numpy 2.0+ compatibility

### Option 4: Alternative Perturbation Prediction (RECOMMENDED)

Use modern alternatives - see [ALTERNATIVES_COMPARISON.md](ALTERNATIVES_COMPARISON.md) for detailed analysis:

**Primary Recommendation:**
- **pertpy + GEARS** - Modern framework with state-of-the-art graph neural network models
  - Python 3.11+ compatible
  - `pip install pertpy && pip install cell-gears`
  - Access to 44 harmonized perturbation datasets
  - Published Nature Methods 2025, Nature Biotech 2024

**Other Options:**
- **Custom scvi-tools VAE** - Maximum flexibility, modern codebase
- **CellOracle** - GRN-based approach for TF perturbations

See [detailed comparison →](ALTERNATIVES_COMPARISON.md)

## What Works Now

Despite the scgen import issue, the following are functional:

1. **Project Structure** ✅
   - All files created correctly
   - Tests written
   - Documentation complete

2. **Core Utilities** ✅
   - `data_loader.py` - GEO loading works (synthetic data)
   - `prediction.py` - DE analysis works
   - `visualization.py` - Plotting functions work

3. **MCP Server** ✅
   - FastMCP tools defined correctly
   - Pydantic models validated
   - Server can start (but tools will error without scgen)

## Testing Workaround

To test the non-scgen components:

```bash
# Test data loader only
pytest tests/test_data_loader.py::TestDatasetLoader::test_load_gse184880_synthetic -v

# Test prediction utilities (mock scgen)
# Would need to create mocks for ScGenWrapper
```

## Recommended Action

**For immediate testing**: Use Option 2 (Docker with Python 3.9)

**For production**: Use Option 3 (Custom scGen implementation with modern scvi-tools)

**Alternative**: Use Option 4 (Switch to modern perturbation prediction framework)

## Attempted Installation Summary

**What was tried:**
1. Python 3.11 + scvi-tools 0.14.6 - scgen requires >=0.15.0 ❌
2. Python 3.11 + scvi-tools 0.16.4 - zarr/numpy/pandas binary conflicts ❌
3. Relaxed version constraints - circular dependency hell ❌

**Conclusion:**
The scgen 2.1.0 package from 2021 cannot be installed in a modern Python 3.11 environment without creating a completely isolated environment (Docker/conda) with Python 3.9 and exact old package versions.

