# Harvey LAB External Run Manifest

- Source: `https://github.com/rushilchugh01/harvey-labs-ablations-and-benchmarks`
- Article: `https://rushilchugh.substack.com/p/what-agent-memory-actually-fixes`
- Generator: `gpt-5.5`
- Judge: `gpt-5.4-mini`
- Task count: `10`

## Doctor

```bash
zaxy harvey-lab-doctor path/to/harvey-zaxy-worktree
```

## Adapter

```bash
zaxy harvey-lab-adapter-kit --output-dir reports/benchmarks/harvey-lab-adapter-kit
```

## Tasks

| Task | Shape | Index | Run | Judge | Normalize | Validate |
|------|-------|-------|-----|-------|-----------|----------|
| corporate-governance/assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements | Compact legal-risk synthesis | `zaxy harvey-lab-index --normalized-corpus-root .ingestion/corpora/corporate-governance__assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements/txt --source-map .ingestion/corpora/corporate-governance__assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements/source-map.json --output-dir .ingestion/indexes/corporate-governance__assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements/zaxy` | `HARVEY_MEMORY_MANIFEST=.ingestion/indexes/corporate-governance__assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements/zaxy/manifest.json uv run python -m harness.run --model gpt-5.5 --task corporate-governance/assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements --run-id zaxy-corporate-governance__assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements --reasoning-effort low` | `uv run python -m evaluation.run_eval --judge-model gpt-5.4-mini --run-id zaxy-corporate-governance__assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements --task corporate-governance/assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements` | `zaxy harvey-lab-normalize-run --harvey-worktree . --run-id zaxy-corporate-governance__assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements --task-id corporate-governance/assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements --manifest .ingestion/indexes/corporate-governance__assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements/zaxy/manifest.json` | `uv run python scripts/memory_ablation/validate_result.py --run-dir .ingestion/runs/zaxy-corporate-governance__assess-impact-of-ftc-noncompete-ban-on-existing-employment-agreements --worktree-root .` |
| corporate-ma/analyze-change-of-control-provisions-across-targets-material-contracts | Sparse clause hunt | `zaxy harvey-lab-index --normalized-corpus-root .ingestion/corpora/corporate-ma__analyze-change-of-control-provisions-across-targets-material-contracts/txt --source-map .ingestion/corpora/corporate-ma__analyze-change-of-control-provisions-across-targets-material-contracts/source-map.json --output-dir .ingestion/indexes/corporate-ma__analyze-change-of-control-provisions-across-targets-material-contracts/zaxy` | `HARVEY_MEMORY_MANIFEST=.ingestion/indexes/corporate-ma__analyze-change-of-control-provisions-across-targets-material-contracts/zaxy/manifest.json uv run python -m harness.run --model gpt-5.5 --task corporate-ma/analyze-change-of-control-provisions-across-targets-material-contracts --run-id zaxy-corporate-ma__analyze-change-of-control-provisions-across-targets-material-contracts --reasoning-effort low` | `uv run python -m evaluation.run_eval --judge-model gpt-5.4-mini --run-id zaxy-corporate-ma__analyze-change-of-control-provisions-across-targets-material-contracts --task corporate-ma/analyze-change-of-control-provisions-across-targets-material-contracts` | `zaxy harvey-lab-normalize-run --harvey-worktree . --run-id zaxy-corporate-ma__analyze-change-of-control-provisions-across-targets-material-contracts --task-id corporate-ma/analyze-change-of-control-provisions-across-targets-material-contracts --manifest .ingestion/indexes/corporate-ma__analyze-change-of-control-provisions-across-targets-material-contracts/zaxy/manifest.json` | `uv run python scripts/memory_ablation/validate_result.py --run-dir .ingestion/runs/zaxy-corporate-ma__analyze-change-of-control-provisions-across-targets-material-contracts --worktree-root .` |
| corporate-ma/draft-acquisition-due-diligence | Broad diligence sweep | `zaxy harvey-lab-index --normalized-corpus-root .ingestion/corpora/corporate-ma__draft-acquisition-due-diligence/txt --source-map .ingestion/corpora/corporate-ma__draft-acquisition-due-diligence/source-map.json --output-dir .ingestion/indexes/corporate-ma__draft-acquisition-due-diligence/zaxy` | `HARVEY_MEMORY_MANIFEST=.ingestion/indexes/corporate-ma__draft-acquisition-due-diligence/zaxy/manifest.json uv run python -m harness.run --model gpt-5.5 --task corporate-ma/draft-acquisition-due-diligence --run-id zaxy-corporate-ma__draft-acquisition-due-diligence --reasoning-effort low` | `uv run python -m evaluation.run_eval --judge-model gpt-5.4-mini --run-id zaxy-corporate-ma__draft-acquisition-due-diligence --task corporate-ma/draft-acquisition-due-diligence` | `zaxy harvey-lab-normalize-run --harvey-worktree . --run-id zaxy-corporate-ma__draft-acquisition-due-diligence --task-id corporate-ma/draft-acquisition-due-diligence --manifest .ingestion/indexes/corporate-ma__draft-acquisition-due-diligence/zaxy/manifest.json` | `uv run python scripts/memory_ablation/validate_result.py --run-dir .ingestion/runs/zaxy-corporate-ma__draft-acquisition-due-diligence --worktree-root .` |
| corporate-ma/review-data-room-red-flag-review | Red-flag spotting | `zaxy harvey-lab-index --normalized-corpus-root .ingestion/corpora/corporate-ma__review-data-room-red-flag-review/txt --source-map .ingestion/corpora/corporate-ma__review-data-room-red-flag-review/source-map.json --output-dir .ingestion/indexes/corporate-ma__review-data-room-red-flag-review/zaxy` | `HARVEY_MEMORY_MANIFEST=.ingestion/indexes/corporate-ma__review-data-room-red-flag-review/zaxy/manifest.json uv run python -m harness.run --model gpt-5.5 --task corporate-ma/review-data-room-red-flag-review --run-id zaxy-corporate-ma__review-data-room-red-flag-review --reasoning-effort low` | `uv run python -m evaluation.run_eval --judge-model gpt-5.4-mini --run-id zaxy-corporate-ma__review-data-room-red-flag-review --task corporate-ma/review-data-room-red-flag-review` | `zaxy harvey-lab-normalize-run --harvey-worktree . --run-id zaxy-corporate-ma__review-data-room-red-flag-review --task-id corporate-ma/review-data-room-red-flag-review --manifest .ingestion/indexes/corporate-ma__review-data-room-red-flag-review/zaxy/manifest.json` | `uv run python scripts/memory_ablation/validate_result.py --run-dir .ingestion/runs/zaxy-corporate-ma__review-data-room-red-flag-review --worktree-root .` |
| data-privacy-cybersecurity/compare-privacy-program-documentation-against-applicable-data-protection-regulations | Compliance mapping | `zaxy harvey-lab-index --normalized-corpus-root .ingestion/corpora/data-privacy-cybersecurity__compare-privacy-program-documentation-against-applicable-data-protection-regulations/txt --source-map .ingestion/corpora/data-privacy-cybersecurity__compare-privacy-program-documentation-against-applicable-data-protection-regulations/source-map.json --output-dir .ingestion/indexes/data-privacy-cybersecurity__compare-privacy-program-documentation-against-applicable-data-protection-regulations/zaxy` | `HARVEY_MEMORY_MANIFEST=.ingestion/indexes/data-privacy-cybersecurity__compare-privacy-program-documentation-against-applicable-data-protection-regulations/zaxy/manifest.json uv run python -m harness.run --model gpt-5.5 --task data-privacy-cybersecurity/compare-privacy-program-documentation-against-applicable-data-protection-regulations --run-id zaxy-data-privacy-cybersecurity__compare-privacy-program-documentation-against-applicable-data-protection-regulations --reasoning-effort low` | `uv run python -m evaluation.run_eval --judge-model gpt-5.4-mini --run-id zaxy-data-privacy-cybersecurity__compare-privacy-program-documentation-against-applicable-data-protection-regulations --task data-privacy-cybersecurity/compare-privacy-program-documentation-against-applicable-data-protection-regulations` | `zaxy harvey-lab-normalize-run --harvey-worktree . --run-id zaxy-data-privacy-cybersecurity__compare-privacy-program-documentation-against-applicable-data-protection-regulations --task-id data-privacy-cybersecurity/compare-privacy-program-documentation-against-applicable-data-protection-regulations --manifest .ingestion/indexes/data-privacy-cybersecurity__compare-privacy-program-documentation-against-applicable-data-protection-regulations/zaxy/manifest.json` | `uv run python scripts/memory_ablation/validate_result.py --run-dir .ingestion/runs/zaxy-data-privacy-cybersecurity__compare-privacy-program-documentation-against-applicable-data-protection-regulations --worktree-root .` |
| litigation-dispute-resolution/build-litigation-case-timeline | Event reconstruction | `zaxy harvey-lab-index --normalized-corpus-root .ingestion/corpora/litigation-dispute-resolution__build-litigation-case-timeline/txt --source-map .ingestion/corpora/litigation-dispute-resolution__build-litigation-case-timeline/source-map.json --output-dir .ingestion/indexes/litigation-dispute-resolution__build-litigation-case-timeline/zaxy` | `HARVEY_MEMORY_MANIFEST=.ingestion/indexes/litigation-dispute-resolution__build-litigation-case-timeline/zaxy/manifest.json uv run python -m harness.run --model gpt-5.5 --task litigation-dispute-resolution/build-litigation-case-timeline --run-id zaxy-litigation-dispute-resolution__build-litigation-case-timeline --reasoning-effort low` | `uv run python -m evaluation.run_eval --judge-model gpt-5.4-mini --run-id zaxy-litigation-dispute-resolution__build-litigation-case-timeline --task litigation-dispute-resolution/build-litigation-case-timeline` | `zaxy harvey-lab-normalize-run --harvey-worktree . --run-id zaxy-litigation-dispute-resolution__build-litigation-case-timeline --task-id litigation-dispute-resolution/build-litigation-case-timeline --manifest .ingestion/indexes/litigation-dispute-resolution__build-litigation-case-timeline/zaxy/manifest.json` | `uv run python scripts/memory_ablation/validate_result.py --run-dir .ingestion/runs/zaxy-litigation-dispute-resolution__build-litigation-case-timeline --worktree-root .` |
| litigation-dispute-resolution/categorize-document-production-set-by-relevance-and-privilege | Document-by-document coding | `zaxy harvey-lab-index --normalized-corpus-root .ingestion/corpora/litigation-dispute-resolution__categorize-document-production-set-by-relevance-and-privilege/txt --source-map .ingestion/corpora/litigation-dispute-resolution__categorize-document-production-set-by-relevance-and-privilege/source-map.json --output-dir .ingestion/indexes/litigation-dispute-resolution__categorize-document-production-set-by-relevance-and-privilege/zaxy` | `HARVEY_MEMORY_MANIFEST=.ingestion/indexes/litigation-dispute-resolution__categorize-document-production-set-by-relevance-and-privilege/zaxy/manifest.json uv run python -m harness.run --model gpt-5.5 --task litigation-dispute-resolution/categorize-document-production-set-by-relevance-and-privilege --run-id zaxy-litigation-dispute-resolution__categorize-document-production-set-by-relevance-and-privilege --reasoning-effort low` | `uv run python -m evaluation.run_eval --judge-model gpt-5.4-mini --run-id zaxy-litigation-dispute-resolution__categorize-document-production-set-by-relevance-and-privilege --task litigation-dispute-resolution/categorize-document-production-set-by-relevance-and-privilege` | `zaxy harvey-lab-normalize-run --harvey-worktree . --run-id zaxy-litigation-dispute-resolution__categorize-document-production-set-by-relevance-and-privilege --task-id litigation-dispute-resolution/categorize-document-production-set-by-relevance-and-privilege --manifest .ingestion/indexes/litigation-dispute-resolution__categorize-document-production-set-by-relevance-and-privilege/zaxy/manifest.json` | `uv run python scripts/memory_ablation/validate_result.py --run-dir .ingestion/runs/zaxy-litigation-dispute-resolution__categorize-document-production-set-by-relevance-and-privilege --worktree-root .` |
| litigation-dispute-resolution/review-document-production-set-for-attorney | Production-set classification | `zaxy harvey-lab-index --normalized-corpus-root .ingestion/corpora/litigation-dispute-resolution__review-document-production-set-for-attorney/txt --source-map .ingestion/corpora/litigation-dispute-resolution__review-document-production-set-for-attorney/source-map.json --output-dir .ingestion/indexes/litigation-dispute-resolution__review-document-production-set-for-attorney/zaxy` | `HARVEY_MEMORY_MANIFEST=.ingestion/indexes/litigation-dispute-resolution__review-document-production-set-for-attorney/zaxy/manifest.json uv run python -m harness.run --model gpt-5.5 --task litigation-dispute-resolution/review-document-production-set-for-attorney --run-id zaxy-litigation-dispute-resolution__review-document-production-set-for-attorney --reasoning-effort low` | `uv run python -m evaluation.run_eval --judge-model gpt-5.4-mini --run-id zaxy-litigation-dispute-resolution__review-document-production-set-for-attorney --task litigation-dispute-resolution/review-document-production-set-for-attorney` | `zaxy harvey-lab-normalize-run --harvey-worktree . --run-id zaxy-litigation-dispute-resolution__review-document-production-set-for-attorney --task-id litigation-dispute-resolution/review-document-production-set-for-attorney --manifest .ingestion/indexes/litigation-dispute-resolution__review-document-production-set-for-attorney/zaxy/manifest.json` | `uv run python scripts/memory_ablation/validate_result.py --run-dir .ingestion/runs/zaxy-litigation-dispute-resolution__review-document-production-set-for-attorney --worktree-root .` |
| litigation-dispute-resolution/review-privilege-log-clawback-review | Large log-heavy classification | `zaxy harvey-lab-index --normalized-corpus-root .ingestion/corpora/litigation-dispute-resolution__review-privilege-log-clawback-review/txt --source-map .ingestion/corpora/litigation-dispute-resolution__review-privilege-log-clawback-review/source-map.json --output-dir .ingestion/indexes/litigation-dispute-resolution__review-privilege-log-clawback-review/zaxy` | `HARVEY_MEMORY_MANIFEST=.ingestion/indexes/litigation-dispute-resolution__review-privilege-log-clawback-review/zaxy/manifest.json uv run python -m harness.run --model gpt-5.5 --task litigation-dispute-resolution/review-privilege-log-clawback-review --run-id zaxy-litigation-dispute-resolution__review-privilege-log-clawback-review --reasoning-effort low` | `uv run python -m evaluation.run_eval --judge-model gpt-5.4-mini --run-id zaxy-litigation-dispute-resolution__review-privilege-log-clawback-review --task litigation-dispute-resolution/review-privilege-log-clawback-review` | `zaxy harvey-lab-normalize-run --harvey-worktree . --run-id zaxy-litigation-dispute-resolution__review-privilege-log-clawback-review --task-id litigation-dispute-resolution/review-privilege-log-clawback-review --manifest .ingestion/indexes/litigation-dispute-resolution__review-privilege-log-clawback-review/zaxy/manifest.json` | `uv run python scripts/memory_ablation/validate_result.py --run-dir .ingestion/runs/zaxy-litigation-dispute-resolution__review-privilege-log-clawback-review --worktree-root .` |
| white-collar-defense-investigations/compare-document-production-set-against-subpoena-request-categories | Request matching | `zaxy harvey-lab-index --normalized-corpus-root .ingestion/corpora/white-collar-defense-investigations__compare-document-production-set-against-subpoena-request-categories/txt --source-map .ingestion/corpora/white-collar-defense-investigations__compare-document-production-set-against-subpoena-request-categories/source-map.json --output-dir .ingestion/indexes/white-collar-defense-investigations__compare-document-production-set-against-subpoena-request-categories/zaxy` | `HARVEY_MEMORY_MANIFEST=.ingestion/indexes/white-collar-defense-investigations__compare-document-production-set-against-subpoena-request-categories/zaxy/manifest.json uv run python -m harness.run --model gpt-5.5 --task white-collar-defense-investigations/compare-document-production-set-against-subpoena-request-categories --run-id zaxy-white-collar-defense-investigations__compare-document-production-set-against-subpoena-request-categories --reasoning-effort low` | `uv run python -m evaluation.run_eval --judge-model gpt-5.4-mini --run-id zaxy-white-collar-defense-investigations__compare-document-production-set-against-subpoena-request-categories --task white-collar-defense-investigations/compare-document-production-set-against-subpoena-request-categories` | `zaxy harvey-lab-normalize-run --harvey-worktree . --run-id zaxy-white-collar-defense-investigations__compare-document-production-set-against-subpoena-request-categories --task-id white-collar-defense-investigations/compare-document-production-set-against-subpoena-request-categories --manifest .ingestion/indexes/white-collar-defense-investigations__compare-document-production-set-against-subpoena-request-categories/zaxy/manifest.json` | `uv run python scripts/memory_ablation/validate_result.py --run-dir .ingestion/runs/zaxy-white-collar-defense-investigations__compare-document-production-set-against-subpoena-request-categories --worktree-root .` |

## Collect

```bash
uv run python scripts/memory_ablation/collect_results.py --worktree . --dedupe-latest --output .ingestion/reports/comparison-zaxy.json
```

## Status

```bash
zaxy harvey-lab-status path/to/harvey-zaxy-worktree
```

## Compare

```bash
zaxy harvey-lab-import path/to/harvey-zaxy-worktree --output-dir reports/benchmarks/harvey-lab-memory-ablation
```

## Validate And Publish

Report JSON: `reports/benchmarks/harvey-lab-memory-ablation/harvey-lab-benchmark.json`

Publish output: `reports/benchmarks/harvey-lab-memory-ablation/publishable-statistics.md`

```bash
zaxy harvey-lab-validate reports/benchmarks/harvey-lab-memory-ablation/harvey-lab-benchmark.json --require-complete
```

```bash
zaxy harvey-lab-gate reports/benchmarks/harvey-lab-memory-ablation/harvey-lab-benchmark.json
```

```bash
zaxy harvey-lab-publish reports/benchmarks/harvey-lab-memory-ablation/harvey-lab-benchmark.json --output reports/benchmarks/harvey-lab-memory-ablation/publishable-statistics.md
```
