# Backup & restore

Legate's durable state lives in four places. Back up all of them together.

| Store | What it holds | Losing it means |
|---|---|---|
| PostgreSQL | Users, workspaces, agents, workflows, runs, steps, approvals, audit log, usage, **encrypted credentials** | Everything except vectors/files |
| Qdrant | RAG vectors | Re-ingest documents to rebuild |
| Object storage | Uploaded document files | Can't re-ingest without originals |
| `LEGATE_MASTER_KEY` | The KEK that unwraps credential data keys | **Stored credentials become undecryptable** |

> ⚠️ The master key is not in the database. Store it in your secrets manager and
> back it up separately — without it, encrypted connector credentials are lost.

## Backup

```bash
# PostgreSQL (compressed custom format).
pg_dump "$DATABASE_URL" -Fc -f legate-$(date +%F).dump

# Qdrant — snapshot every collection via its API, or back up the storage volume.
curl -X POST "$QDRANT_URL/collections/{collection}/snapshots"

# Object storage — the local volume (or use your S3 bucket's versioning).
tar czf legate-storage-$(date +%F).tgz ./data/storage
```

Automate with a CronJob / scheduled task; keep the master key wherever you keep
other secrets.

## Restore

```bash
# 1. Bring up a fresh stack (empty Postgres/Qdrant), with the SAME LEGATE_MASTER_KEY.

# 2. Restore PostgreSQL.
pg_restore --clean --if-exists -d "$DATABASE_URL" legate-YYYY-MM-DD.dump

# 3. Restore object storage.
tar xzf legate-storage-YYYY-MM-DD.tgz -C ./data/storage

# 4. Restore Qdrant from a snapshot, or (simplest) re-ingest:
#    delete + re-upload each document, or POST /documents/{id}/reingest.

# 5. Verify.
curl https://$DOMAIN/api/health
# Then confirm the audit chain is intact and a connector still decrypts:
#   POST /connectors/{id}/test
```

A restore is complete when `/health` is green, `POST /connectors/{id}/test`
succeeds (proving the master key still unwraps credentials), and the audit log
verifies (`legate.audit.service.verify_chain`).
