# S3 Project Storage (Legacy)

> **Note**: S3 is no longer used for project hibernation. The system now uses **EBS VolumeSnapshots** for project persistence. This infrastructure exists for potential future use (asset storage, logs, etc.) but is not actively used for the hibernation workflow.
>
> See [snapshot-manager.md](../../orchestrator/services/snapshot-manager.md) for the current persistence approach.

Amazon S3 bucket originally used for project hibernation and restoration.

**File**: `c:/Users/Smirk/Downloads/Tesslate-Studio/k8s/terraform/aws/s3.tf`

## Bucket Configuration

**Name**: `tesslate-projects-production-{random-suffix}`
- Suffix prevents naming conflicts
- Example: `tesslate-projects-production-7761157a`

**Region**: us-east-1
**Access**: Private (public access blocked)

## Features

### Versioning

**Enabled**: Protects against accidental deletion

Every PUT creates a new version. Old versions retained per lifecycle rules.

**View versions**:
```bash
aws s3api list-object-versions --bucket tesslate-projects-production-7761157a --prefix projects/
```

**Restore deleted object**:
```bash
aws s3api delete-object --bucket {bucket} --key {key} --version-id {delete-marker-version-id}
```

### Encryption

**Type**: AES256 (SSE-S3)
**Bucket Key**: Enabled (reduces encryption costs)

All objects automatically encrypted at rest.

### Public Access Block

All public access blocked:
- `block_public_acls = true`
- `block_public_policy = true`
- `ignore_public_acls = true`
- `restrict_public_buckets = true`

Bucket is only accessible via:
- IAM roles (IRSA for backend pods)
- Signed URLs (for frontend uploads)

### Lifecycle Rules

#### Rule 1: Archive Old Versions

**Target**: Non-current versions (old versions of objects)

**Transitions**:
1. After 30 days → STANDARD_IA (Infrequent Access, 50% cheaper)
2. After 90 days → GLACIER (Deep archive, 90% cheaper)
3. After 365 days → Expire (delete)

**Cost Savings**: Old project versions moved to cheaper storage.

#### Rule 2: Clean Up Multipart Uploads

**Target**: Incomplete multipart uploads

**Action**: Abort after 7 days

**Reason**: Failed uploads leave orphaned parts, waste storage.

#### Rule 3: Expire Inactive Projects (Optional, Commented Out)

**Target**: Objects in `projects/` prefix not accessed for 180 days

**Action**: Delete

**Note**: Currently commented out. Enable for automatic cleanup:
```hcl
rule {
  id     = "expire-inactive-projects"
  status = "Enabled"

  filter {
    prefix = "projects/"
  }

  expiration {
    days = 180
  }
}
```

### CORS Configuration

**Allowed Origins**:
- `https://your-domain.com`
- `https://*.your-domain.com`

**Allowed Methods**: GET, PUT, POST, DELETE, HEAD

**Allowed Headers**: `*`

**Exposed Headers**: ETag

**Max Age**: 3600 seconds (1 hour)

**Purpose**: Allow frontend to upload directly to S3 (signed URLs).

### Bucket Policy

**Enforce TLS**:

Denies all requests over HTTP (unencrypted). Only HTTPS allowed.

```json
{
  "Sid": "EnforceTLS",
  "Effect": "Deny",
  "Principal": "*",
  "Action": "s3:*",
  "Condition": {
    "Bool": {
      "aws:SecureTransport": "false"
    }
  }
}
```

## Usage

### Upload Project (Backend via IRSA)

```python
import boto3

s3 = boto3.client('s3')
s3.upload_file('/tmp/project.zip', 'tesslate-projects-production-7761157a', 'projects/abc123.zip')
```

**No credentials needed** - IRSA provides automatic authentication.

### Download Project (Backend via IRSA)

```python
s3.download_file('tesslate-projects-production-7761157a', 'projects/abc123.zip', '/tmp/project.zip')
```

### List Projects

```bash
aws s3 ls s3://tesslate-projects-production-7761157a/projects/
```

### Check Size

```bash
aws s3 ls s3://tesslate-projects-production-7761157a/projects/ --recursive --human-readable --summarize
```

## Storage Classes

**STANDARD** (default):
- $0.023/GB/month
- Instant retrieval
- Use for: Active projects

**STANDARD_IA** (Infrequent Access):
- $0.0125/GB/month
- Instant retrieval
- Use for: Old project versions (30+ days)

**GLACIER**:
- $0.004/GB/month
- Retrieval: minutes to hours
- Use for: Very old versions (90+ days)

**Lifecycle transitions** automatically move objects between classes.

## Monitoring

### Bucket Size

```bash
aws s3 ls s3://tesslate-projects-production-7761157a --recursive --summarize | grep "Total Size"
```

### Object Count

```bash
aws s3 ls s3://tesslate-projects-production-7761157a --recursive --summarize | grep "Total Objects"
```

### CloudWatch Metrics

- BucketSizeBytes
- NumberOfObjects
- AllRequests
- GetRequests
- PutRequests

## Cost Estimation

**Example**: 100 active projects, 1GB each (compressed)

**Active projects** (STANDARD):
- 100 × 1GB × $0.023/GB = $2.30/month

**Old versions** (30-90 days, STANDARD_IA):
- 200 × 0.5GB × $0.0125/GB = $1.25/month

**Archive** (90+ days, GLACIER):
- 500 × 0.3GB × $0.004/GB = $0.60/month

**Total**: ~$4.15/month for 100 projects + history

## Troubleshooting

### Access Denied

**Check IRSA**:
```bash
kubectl describe sa -n tesslate tesslate-backend-sa
# Should have annotation: eks.amazonaws.com/role-arn
```

**Check IAM role**:
```bash
aws iam get-role --role-name tesslate-backend-s3-access
```

### Object Not Found

**Check key name**:
```bash
aws s3 ls s3://tesslate-projects-production-7761157a/projects/{project-id}.zip
```

**Check versions** (if versioning enabled):
```bash
aws s3api list-object-versions --bucket tesslate-projects-production-7761157a --prefix projects/{project-id}.zip
```

### Slow Uploads/Downloads

**Increase concurrency** (default: 10):
```bash
aws configure set default.s3.max_concurrent_requests 20
```

**Use multipart uploads** for files >100MB:
```python
s3.upload_file(filename, bucket, key, Config=boto3.s3.transfer.TransferConfig(multipart_threshold=100*1024*1024))
```

## Security Best Practices

1. **Never make bucket public**: Already blocked by default
2. **Use IRSA, not credentials**: No keys in code or config
3. **Enable versioning**: Protect against accidental deletion
4. **Enforce TLS**: Bucket policy denies HTTP
5. **Monitor access**: CloudTrail logs all S3 operations
6. **Least privilege**: IAM policies grant only necessary permissions

## Disaster Recovery

### Enable Cross-Region Replication (Optional)

**Purpose**: Replicate critical projects to another region

**Configuration**:
```hcl
resource "aws_s3_bucket_replication_configuration" "replication" {
  bucket = aws_s3_bucket.tesslate_projects.id

  rule {
    id     = "replicate-all"
    status = "Enabled"

    destination {
      bucket        = aws_s3_bucket.replica.arn
      storage_class = "STANDARD_IA"
    }
  }
}
```

### Backup Critical Projects

```bash
# Sync to backup bucket
aws s3 sync s3://tesslate-projects-production-7761157a/projects/ s3://tesslate-backups/projects/
```

## Related Documentation

- [README.md](README.md): Terraform overview
- [eks.md](eks.md): EKS cluster
- [ecr.md](ecr.md): Container registry
- [../../orchestrator/services/snapshot-manager.md](../../orchestrator/services/snapshot-manager.md): Current persistence approach (EBS VolumeSnapshots)
- AWS S3: https://docs.aws.amazon.com/s3/
