TEST 3: Spatial Transcriptomics Analysis (SYNTHETIC_DATA Mode)
==============================================================

> **Data Mode:** This test uses **SYNTHETIC_DATA** — `*_DRY_RUN=false`. The spatialtools server parses the actual CSV files in `/data/patient-data/PAT002-BC-2026/spatial/`. No heavy bioinformatics tools required, but Python parsing dependencies (pandas, numpy, scipy) must be installed. See [Data Modes Guide](../../data-modes-guide.md) for details.

Patient ID: PAT002-BC-2026

**This test uses TABULAR DATA (CSV files), NOT images.**
Use ONLY the spatialtools server. DeepCell is NOT needed.

## Prerequisites

| Requirement | Details |
|------------|---------|
| `SPATIALTOOLS_DRY_RUN` | `false` |
| Python deps | pandas, numpy, scipy (for CSV parsing, spatial stats) |
| Data files | `data/patient-data/PAT002-BC-2026/spatial/` |

## Data Files (Real CSVs)

This test uses the **patient-prefixed** CSV files (generated by `generate_PAT002_missing_files.py`):
- `PAT002_expression.csv` — Patient-prefixed gene expression (900 spots, 36 genes, luminal A profile)
- `PAT002_coordinates.csv` — Patient-prefixed spatial coordinates with patient_id column
- `PAT002_regions.csv` — Patient-prefixed tissue region annotations (5 BC regions)

Also available (original Visium format, unprefixed):
- `visium_gene_expression.csv` — Original 900-spot expression matrix
- `visium_spatial_coordinates.csv` — Original spot coordinates
- `visium_region_annotations.csv` — Original 7-region annotations

## Analysis Steps

### Pre-Flight Validation

1. **File existence and integrity check:**
   - Confirm patient-prefixed CSV files are accessible and parseable
   - Spot IDs should use `PAT002_SPOT_*` prefix format
   - All files should contain a `patient_id` column with value `PAT002-BC-2026`

2. **Data consistency check:**
   - Same number of spots across all 3 files (900)
   - Expected genes present in expression file
   - Expected regions present in annotations

3. **Expected genes present:**
   - Verify key breast cancer genes: ESR1, PGR, GATA3, FOXA1, MKI67, CCND1, PIK3CA, CD8A, CD68, VIM, KRT18, EPCAM

4. **Expected regions (5 breast tissue types):**
   - `tumor_core` (~305 spots)
   - `adipose` (~287 spots)
   - `stroma` (~186 spots)
   - `immune_infiltrate` (~118 spots)
   - `normal_epithelium` (~4 spots)

### Spatial Analysis

1. **Load spatial data:**
   ```
   Use mcp-spatialtools to load from:
   - data/patient-data/PAT002-BC-2026/spatial/PAT002_expression.csv
   - data/patient-data/PAT002-BC-2026/spatial/PAT002_coordinates.csv
   - data/patient-data/PAT002-BC-2026/spatial/PAT002_regions.csv
   ```
   - How many spatial spots?
   - What spatial regions are identified?
   - How many spots per region?

2. **Focus on KEY GENES (breast cancer panel):**
   - **Hormone receptors:** ESR1, PGR, GATA3, FOXA1
   - **Proliferation:** MKI67, PCNA, TOP2A
   - **PI3K pathway:** PIK3CA, AKT1, MTOR
   - **Immune:** CD3D, CD8A, CD4, FOXP3, CD68, CD163
   - **Stromal:** VIM, COL1A1, ACTA2, FAP
   - **Epithelial:** KRT8, KRT18, KRT19, EPCAM

3. **Luminal A profile validation:**
   - Confirm ESR1, PGR, GATA3, FOXA1 are elevated (luminal signature boosted by generator)
   - Confirm ERBB2 (HER2) is NOT elevated (values 0-5, HER2-negative)
   - This is a key biological validation of the synthetic data

4. **Gene expression by region:**
   - Which regions have high ER/PR expression? (expect `tumor_core`)
   - Where are proliferation markers (MKI67) highest?
   - Where are immune cells (CD8A, CD68) located? (expect `immune_infiltrate`)
   - Stromal marker distribution? (expect `stroma`)

5. **Spatial patterns:**
   - Spatial heterogeneity in hormone receptor expression?
   - Are immune cells infiltrating the tumor or excluded?
   - Tumor-stroma interface patterns?
   - B cell clustering suggestive of tertiary lymphoid structures (TLS)?

6. **Generate visualizations:**
   ```
   Tool: generate_spatial_heatmap
   Inputs:
   - expression_file: PAT002_expression.csv
   - coordinates_file: PAT002_coordinates.csv
   - genes: ["ESR1", "PGR", "MKI67", "PIK3CA", "CD8A", "CD68"]
   ```

   ```
   Tool: generate_region_composition_chart
   Inputs:
   - regions_file: PAT002_regions.csv
   ```

   ```
   Tool: generate_gene_expression_heatmap
   Inputs:
   - expression_file: PAT002_expression.csv
   - regions_file: PAT002_regions.csv
   - genes: ["ESR1", "PGR", "GATA3", "FOXA1", "MKI67", "PIK3CA", "CD8A", "CD68", "VIM"]
   ```

## How This Differs from DRY_RUN

| Aspect | DRY_RUN | SYNTHETIC_DATA (this test) |
|--------|---------|---------------------------|
| Data source | Hardcoded inline values | Parsed from actual CSVs on disk |
| CSV files | `visium_*.csv` (any path) | `PAT002_expression.csv`, `PAT002_coordinates.csv`, `PAT002_regions.csv` |
| Spot IDs | Generic `SPOT_*` | Patient-prefixed `PAT002_SPOT_*` |
| patient_id column | Not present | Present in all 3 files (`PAT002-BC-2026`) |
| Spot counts | Returns fixed 900 | Counts actual rows in CSV |
| Region labels | Returns 6-7 regions | 5 regions mapped from original 7 |
| Expression values | Returns predetermined means | Calculated from actual expression matrix |
| HER2 validation | Not tested | ERBB2 values confirmed low (0-5) |
| Moran's I | Returns fixed statistics | Computes from real spatial coordinates + expression |
| File I/O | None | Reads 3 CSV files from `data/` directory |

## Expected Results

Results come from parsing the actual synthetic CSV files:

**Spatial Structure:**
- Total spots: 900 (parsed from CSV)
- 5 regions: tumor_core (305), adipose (287), stroma (186), immune_infiltrate (118), normal_epithelium (4)

**Expression Patterns:**

| Gene | High Expression Region | Pattern |
|------|----------------------|---------|
| ESR1 | tumor_core | Strong ER expression (boosted, luminal A) |
| PGR | tumor_core | Strong PR expression (boosted) |
| GATA3 | tumor_core | Luminal lineage marker (boosted) |
| FOXA1 | tumor_core | Luminal pioneer factor (boosted) |
| ERBB2 | None (all low) | HER2-negative confirmed (values 0-5) |
| MKI67 | tumor_core (moderate) | Low-moderate proliferation |
| PIK3CA | tumor_core | PI3K activation |
| CD8A | immune_infiltrate | T cell presence |
| CD68 | immune_infiltrate, stroma | Macrophages |
| VIM | stroma | Stromal marker |

**Spatial Findings:**
- Hormone receptors: Strongly expressed in tumor_core (consistent with ER+/PR+ status)
- HER2 (ERBB2): Confirmed low across all regions — validates HER2-negative
- Proliferation: Moderate, heterogeneous
- Immune cells: Present in immune_infiltrate region (warm TME)
- B cell clustering: Check for spatial co-localization suggestive of TLS

## Output Format

Please provide:

1. **Spatial Structure** — Spot counts and 5 regions from parsed PAT002_*.csv
2. **Gene Expression Heatmap** — Key genes x regions from actual data
3. **Luminal A Validation** — ESR1/PGR/GATA3/FOXA1 elevated, ERBB2 low
4. **Key Spatial Findings** — ER/PR distribution, proliferation, immune patterns
5. **Visualizations** — Spatial heatmaps, region charts, autocorrelation
6. **Clinical Interpretation** — Heterogeneity, immune status, TME classification

## Validation Checkpoints

- [ ] Server correctly reads patient-prefixed CSVs from `data/patient-data/PAT002-BC-2026/spatial/`
- [ ] Spot count matches actual CSV row count (900, not hardcoded)
- [ ] Spot IDs use `PAT002_SPOT_*` prefix
- [ ] patient_id column present and equals `PAT002-BC-2026`
- [ ] 5 region labels read from annotations (tumor_core, adipose, stroma, immune_infiltrate, normal_epithelium)
- [ ] ESR1, PGR, GATA3, FOXA1 expression elevated (luminal A profile)
- [ ] ERBB2 expression low across all regions (HER2-negative spatial confirmation)
- [ ] Expression values are real numbers from CSV (not predetermined)
- [ ] Moran's I calculated from actual spatial data
- [ ] Biological patterns (immune infiltration, tumor ER expression) preserved
