# Fully Parallel Evaluation Results

Generated: 2026-01-11T06:48:57.425Z
Total Time: 300.6s
Suite: pack
Models: 8
Scenarios: 24 of 24 (limit=0)
Total evaluations: 192

## Summary by Model

| Model | Total | Passed | Failed | Avg Time (s) |
|-------|-------|--------|--------|--------------|
| claude-haiku-4.5 | 24 | 24 | 0 | 60.5 |
| gpt-5-mini | 24 | 23 | 0 | 84.0 |
| gemini-3-flash | 24 | 24 | 0 | 14.1 |
| deepseek-r1 | 24 | 23 | 0 | 104.2 |
| deepseek-v3.2 | 24 | 21 | 0 | 129.9 |
| qwen3-235b | 24 | 24 | 0 | 111.7 |
| minimax-m2.1 | 24 | 24 | 0 | 75.6 |
| mimo-v2-flash-free | 24 | 24 | 0 | 52.4 |

## Summary by Scenario

| Scenario | Total | Passed | Failed |
|----------|-------|--------|--------|
| Next: academic tool-driven literature debrief | 8 | 8 | 0 |
| Next: academic vague anchor | 7 | 7 | 0 |
| Next: banker tool-driven outbound pack | 8 | 8 | 0 |
| Next: banker vague (fast debrief) | 8 | 8 | 0 |
| Next: CTO tool-driven CVE plan | 8 | 8 | 0 |
| Next: CTO vague exposure | 8 | 8 | 0 |
| Next: ecosystem tool-driven second-order brief | 8 | 8 | 0 |
| Next: ecosystem vague incident | 8 | 8 | 0 |
| Next: exec tool-driven cost model | 8 | 8 | 0 |
| Next: exec vague standardize | 8 | 8 | 0 |
| Next: founder tool-driven memo | 7 | 7 | 0 |
| Next: founder vague positioning | 8 | 8 | 0 |
| Next: product tool-driven expandable card schema | 7 | 7 | 0 |
| Next: product vague UI usable | 8 | 8 | 0 |
| Next: quant tool-driven signal set JSON | 8 | 8 | 0 |
| Next: quant vague what to track | 8 | 8 | 0 |
| Next: sales tool-driven one-screen + objections | 8 | 8 | 0 |
| Next: sales vague shareable | 8 | 8 | 0 |
| Next: VC tool-driven comps + diligence | 7 | 7 | 0 |
| Next: VC vague wedge | 8 | 8 | 0 |
| offset:17 | 1 | 0 | 1 |
| offset:22 | 1 | 0 | 1 |
| offset:3 | 1 | 0 | 1 |
| offset:7 | 1 | 0 | 1 |
| offset:8 | 1 | 0 | 1 |
| Pack: exec cross-provider pricing comparison | 7 | 7 | 0 |
| Pack: meta budgeted deep dive (<=3 tools, <=$0.25) | 8 | 8 | 0 |
| Stress: ambiguous persona (wedge + outreach) | 8 | 8 | 0 |
| Stress: contradiction handling (Seed vs Series A) | 8 | 8 | 0 |

## Detailed Results

### claude-haiku-4.5

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Next: banker vague (fast debrief) | ✅ PASS | 17.5 | - |
| Next: banker tool-driven outbound pack | ✅ PASS | 59.1 | - |
| Next: VC vague wedge | ✅ PASS | 58.8 | - |
| Next: VC tool-driven comps + diligence | ✅ PASS | 52.9 | - |
| Next: CTO vague exposure | ✅ PASS | 55.8 | - |
| Next: CTO tool-driven CVE plan | ✅ PASS | 129.0 | - |
| Next: founder vague positioning | ✅ PASS | 59.1 | - |
| Next: founder tool-driven memo | ✅ PASS | 58.7 | - |
| Next: academic vague anchor | ✅ PASS | 52.8 | - |
| Next: academic tool-driven literature debrief | ✅ PASS | 58.7 | - |
| Next: exec vague standardize | ✅ PASS | 64.8 | - |
| Next: exec tool-driven cost model | ✅ PASS | 56.0 | - |
| Next: ecosystem vague incident | ✅ PASS | 59.0 | - |
| Next: ecosystem tool-driven second-order brief | ✅ PASS | 58.9 | - |
| Next: quant vague what to track | ✅ PASS | 52.6 | - |
| Next: quant tool-driven signal set JSON | ✅ PASS | 52.9 | - |
| Next: product vague UI usable | ✅ PASS | 58.8 | - |
| Next: product tool-driven expandable card schema | ✅ PASS | 113.6 | - |
| Next: sales vague shareable | ✅ PASS | 53.7 | - |
| Next: sales tool-driven one-screen + objections | ✅ PASS | 54.2 | - |
| Stress: ambiguous persona (wedge + outreach) | ✅ PASS | 58.5 | - |
| Stress: contradiction handling (Seed vs Series A) | ✅ PASS | 52.8 | - |
| Pack: exec cross-provider pricing comparison | ✅ PASS | 56.7 | - |
| Pack: meta budgeted deep dive (<=3 tools, <=$0.25) | ✅ PASS | 56.7 | - |

### gpt-5-mini

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Next: banker vague (fast debrief) | ✅ PASS | 65.0 | - |
| Next: banker tool-driven outbound pack | ✅ PASS | 68.9 | - |
| Next: VC vague wedge | ✅ PASS | 70.8 | - |
| Next: VC tool-driven comps + diligence | ✅ PASS | 61.9 | - |
| Next: CTO vague exposure | ✅ PASS | 47.2 | - |
| Next: CTO tool-driven CVE plan | ✅ PASS | 95.1 | - |
| Next: founder vague positioning | ✅ PASS | 69.5 | - |
| Next: founder tool-driven memo | ✅ PASS | 81.3 | - |
| offset:8 | ❌ FAIL | 300.4 | - |
| Next: academic tool-driven literature debrief | ✅ PASS | 134.9 | - |
| Next: exec vague standardize | ✅ PASS | 49.6 | - |
| Next: exec tool-driven cost model | ✅ PASS | 84.2 | - |
| Next: ecosystem vague incident | ✅ PASS | 46.8 | - |
| Next: ecosystem tool-driven second-order brief | ✅ PASS | 65.2 | - |
| Next: quant vague what to track | ✅ PASS | 58.2 | - |
| Next: quant tool-driven signal set JSON | ✅ PASS | 75.8 | - |
| Next: product vague UI usable | ✅ PASS | 66.4 | - |
| Next: product tool-driven expandable card schema | ✅ PASS | 86.9 | - |
| Next: sales vague shareable | ✅ PASS | 56.9 | - |
| Next: sales tool-driven one-screen + objections | ✅ PASS | 53.7 | - |
| Stress: ambiguous persona (wedge + outreach) | ✅ PASS | 59.0 | - |
| Stress: contradiction handling (Seed vs Series A) | ✅ PASS | 74.9 | - |
| Pack: exec cross-provider pricing comparison | ✅ PASS | 168.8 | - |
| Pack: meta budgeted deep dive (<=3 tools, <=$0.25) | ✅ PASS | 74.3 | - |

### gemini-3-flash

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Next: banker vague (fast debrief) | ✅ PASS | 12.8 | - |
| Next: banker tool-driven outbound pack | ✅ PASS | 13.1 | - |
| Next: VC vague wedge | ✅ PASS | 13.6 | - |
| Next: VC tool-driven comps + diligence | ✅ PASS | 13.7 | - |
| Next: CTO vague exposure | ✅ PASS | 12.8 | - |
| Next: CTO tool-driven CVE plan | ✅ PASS | 13.3 | - |
| Next: founder vague positioning | ✅ PASS | 13.8 | - |
| Next: founder tool-driven memo | ✅ PASS | 27.5 | - |
| Next: academic vague anchor | ✅ PASS | 13.5 | - |
| Next: academic tool-driven literature debrief | ✅ PASS | 13.5 | - |
| Next: exec vague standardize | ✅ PASS | 13.5 | - |
| Next: exec tool-driven cost model | ✅ PASS | 13.4 | - |
| Next: ecosystem vague incident | ✅ PASS | 12.9 | - |
| Next: ecosystem tool-driven second-order brief | ✅ PASS | 13.1 | - |
| Next: quant vague what to track | ✅ PASS | 13.3 | - |
| Next: quant tool-driven signal set JSON | ✅ PASS | 12.9 | - |
| Next: product vague UI usable | ✅ PASS | 13.4 | - |
| Next: product tool-driven expandable card schema | ✅ PASS | 13.6 | - |
| Next: sales vague shareable | ✅ PASS | 12.7 | - |
| Next: sales tool-driven one-screen + objections | ✅ PASS | 15.8 | - |
| Stress: ambiguous persona (wedge + outreach) | ✅ PASS | 13.3 | - |
| Stress: contradiction handling (Seed vs Series A) | ✅ PASS | 13.7 | - |
| Pack: exec cross-provider pricing comparison | ✅ PASS | 14.6 | - |
| Pack: meta budgeted deep dive (<=3 tools, <=$0.25) | ✅ PASS | 13.9 | - |

### deepseek-r1

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Next: banker vague (fast debrief) | ✅ PASS | 60.1 | - |
| Next: banker tool-driven outbound pack | ✅ PASS | 162.2 | - |
| Next: VC vague wedge | ✅ PASS | 17.3 | - |
| Next: VC tool-driven comps + diligence | ✅ PASS | 233.4 | - |
| Next: CTO vague exposure | ✅ PASS | 71.9 | - |
| Next: CTO tool-driven CVE plan | ✅ PASS | 225.9 | - |
| Next: founder vague positioning | ✅ PASS | 61.1 | - |
| Next: founder tool-driven memo | ✅ PASS | 59.1 | - |
| Next: academic vague anchor | ✅ PASS | 22.9 | - |
| Next: academic tool-driven literature debrief | ✅ PASS | 156.0 | - |
| Next: exec vague standardize | ✅ PASS | 97.8 | - |
| Next: exec tool-driven cost model | ✅ PASS | 55.4 | - |
| Next: ecosystem vague incident | ✅ PASS | 81.8 | - |
| Next: ecosystem tool-driven second-order brief | ✅ PASS | 13.9 | - |
| Next: quant vague what to track | ✅ PASS | 78.0 | - |
| Next: quant tool-driven signal set JSON | ✅ PASS | 149.7 | - |
| Next: product vague UI usable | ✅ PASS | 57.5 | - |
| Next: product tool-driven expandable card schema | ✅ PASS | 22.7 | - |
| Next: sales vague shareable | ✅ PASS | 106.5 | - |
| Next: sales tool-driven one-screen + objections | ✅ PASS | 132.8 | - |
| Stress: ambiguous persona (wedge + outreach) | ✅ PASS | 101.2 | - |
| Stress: contradiction handling (Seed vs Series A) | ✅ PASS | 91.8 | - |
| offset:22 | ❌ FAIL | 300.5 | - |
| Pack: meta budgeted deep dive (<=3 tools, <=$0.25) | ✅ PASS | 142.2 | - |

### deepseek-v3.2

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Next: banker vague (fast debrief) | ✅ PASS | 73.0 | - |
| Next: banker tool-driven outbound pack | ✅ PASS | 88.6 | - |
| Next: VC vague wedge | ✅ PASS | 117.8 | - |
| offset:3 | ❌ FAIL | 300.5 | - |
| Next: CTO vague exposure | ✅ PASS | 72.2 | - |
| Next: CTO tool-driven CVE plan | ✅ PASS | 79.9 | - |
| Next: founder vague positioning | ✅ PASS | 92.7 | - |
| offset:7 | ❌ FAIL | 300.5 | - |
| Next: academic vague anchor | ✅ PASS | 121.9 | - |
| Next: academic tool-driven literature debrief | ✅ PASS | 167.4 | - |
| Next: exec vague standardize | ✅ PASS | 48.8 | - |
| Next: exec tool-driven cost model | ✅ PASS | 37.5 | - |
| Next: ecosystem vague incident | ✅ PASS | 79.4 | - |
| Next: ecosystem tool-driven second-order brief | ✅ PASS | 57.4 | - |
| Next: quant vague what to track | ✅ PASS | 106.0 | - |
| Next: quant tool-driven signal set JSON | ✅ PASS | 259.2 | - |
| Next: product vague UI usable | ✅ PASS | 149.3 | - |
| offset:17 | ❌ FAIL | 300.5 | - |
| Next: sales vague shareable | ✅ PASS | 80.7 | - |
| Next: sales tool-driven one-screen + objections | ✅ PASS | 92.5 | - |
| Stress: ambiguous persona (wedge + outreach) | ✅ PASS | 71.3 | - |
| Stress: contradiction handling (Seed vs Series A) | ✅ PASS | 93.4 | - |
| Pack: exec cross-provider pricing comparison | ✅ PASS | 259.3 | - |
| Pack: meta budgeted deep dive (<=3 tools, <=$0.25) | ✅ PASS | 66.8 | - |

### qwen3-235b

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Next: banker vague (fast debrief) | ✅ PASS | 117.8 | - |
| Next: banker tool-driven outbound pack | ✅ PASS | 126.0 | - |
| Next: VC vague wedge | ✅ PASS | 160.2 | - |
| Next: VC tool-driven comps + diligence | ✅ PASS | 129.0 | - |
| Next: CTO vague exposure | ✅ PASS | 87.2 | - |
| Next: CTO tool-driven CVE plan | ✅ PASS | 88.7 | - |
| Next: founder vague positioning | ✅ PASS | 130.1 | - |
| Next: founder tool-driven memo | ✅ PASS | 164.2 | - |
| Next: academic vague anchor | ✅ PASS | 139.5 | - |
| Next: academic tool-driven literature debrief | ✅ PASS | 38.2 | - |
| Next: exec vague standardize | ✅ PASS | 144.3 | - |
| Next: exec tool-driven cost model | ✅ PASS | 31.8 | - |
| Next: ecosystem vague incident | ✅ PASS | 160.9 | - |
| Next: ecosystem tool-driven second-order brief | ✅ PASS | 114.9 | - |
| Next: quant vague what to track | ✅ PASS | 117.8 | - |
| Next: quant tool-driven signal set JSON | ✅ PASS | 140.8 | - |
| Next: product vague UI usable | ✅ PASS | 88.5 | - |
| Next: product tool-driven expandable card schema | ✅ PASS | 184.8 | - |
| Next: sales vague shareable | ✅ PASS | 43.8 | - |
| Next: sales tool-driven one-screen + objections | ✅ PASS | 136.1 | - |
| Stress: ambiguous persona (wedge + outreach) | ✅ PASS | 38.5 | - |
| Stress: contradiction handling (Seed vs Series A) | ✅ PASS | 121.2 | - |
| Pack: exec cross-provider pricing comparison | ✅ PASS | 33.9 | - |
| Pack: meta budgeted deep dive (<=3 tools, <=$0.25) | ✅ PASS | 142.3 | - |

### minimax-m2.1

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Next: banker vague (fast debrief) | ✅ PASS | 60.9 | - |
| Next: banker tool-driven outbound pack | ✅ PASS | 27.3 | - |
| Next: VC vague wedge | ✅ PASS | 42.0 | - |
| Next: VC tool-driven comps + diligence | ✅ PASS | 102.3 | - |
| Next: CTO vague exposure | ✅ PASS | 39.8 | - |
| Next: CTO tool-driven CVE plan | ✅ PASS | 153.8 | - |
| Next: founder vague positioning | ✅ PASS | 101.2 | - |
| Next: founder tool-driven memo | ✅ PASS | 155.4 | - |
| Next: academic vague anchor | ✅ PASS | 115.7 | - |
| Next: academic tool-driven literature debrief | ✅ PASS | 56.8 | - |
| Next: exec vague standardize | ✅ PASS | 48.1 | - |
| Next: exec tool-driven cost model | ✅ PASS | 102.2 | - |
| Next: ecosystem vague incident | ✅ PASS | 49.8 | - |
| Next: ecosystem tool-driven second-order brief | ✅ PASS | 129.5 | - |
| Next: quant vague what to track | ✅ PASS | 33.2 | - |
| Next: quant tool-driven signal set JSON | ✅ PASS | 45.5 | - |
| Next: product vague UI usable | ✅ PASS | 23.6 | - |
| Next: product tool-driven expandable card schema | ✅ PASS | 88.4 | - |
| Next: sales vague shareable | ✅ PASS | 27.5 | - |
| Next: sales tool-driven one-screen + objections | ✅ PASS | 40.7 | - |
| Stress: ambiguous persona (wedge + outreach) | ✅ PASS | 18.9 | - |
| Stress: contradiction handling (Seed vs Series A) | ✅ PASS | 35.9 | - |
| Pack: exec cross-provider pricing comparison | ✅ PASS | 175.4 | - |
| Pack: meta budgeted deep dive (<=3 tools, <=$0.25) | ✅ PASS | 141.1 | - |

### mimo-v2-flash-free

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Next: banker vague (fast debrief) | ✅ PASS | 24.8 | - |
| Next: banker tool-driven outbound pack | ✅ PASS | 36.1 | - |
| Next: VC vague wedge | ✅ PASS | 26.2 | - |
| Next: VC tool-driven comps + diligence | ✅ PASS | 132.9 | - |
| Next: CTO vague exposure | ✅ PASS | 25.0 | - |
| Next: CTO tool-driven CVE plan | ✅ PASS | 122.0 | - |
| Next: founder vague positioning | ✅ PASS | 53.8 | - |
| Next: founder tool-driven memo | ✅ PASS | 96.9 | - |
| Next: academic vague anchor | ✅ PASS | 39.7 | - |
| Next: academic tool-driven literature debrief | ✅ PASS | 132.2 | - |
| Next: exec vague standardize | ✅ PASS | 26.3 | - |
| Next: exec tool-driven cost model | ✅ PASS | 117.0 | - |
| Next: ecosystem vague incident | ✅ PASS | 33.5 | - |
| Next: ecosystem tool-driven second-order brief | ✅ PASS | 54.0 | - |
| Next: quant vague what to track | ✅ PASS | 21.1 | - |
| Next: quant tool-driven signal set JSON | ✅ PASS | 24.7 | - |
| Next: product vague UI usable | ✅ PASS | 25.2 | - |
| Next: product tool-driven expandable card schema | ✅ PASS | 28.6 | - |
| Next: sales vague shareable | ✅ PASS | 19.7 | - |
| Next: sales tool-driven one-screen + objections | ✅ PASS | 22.5 | - |
| Stress: ambiguous persona (wedge + outreach) | ✅ PASS | 19.3 | - |
| Stress: contradiction handling (Seed vs Series A) | ✅ PASS | 29.2 | - |
| Pack: exec cross-provider pricing comparison | ✅ PASS | 84.9 | - |
| Pack: meta budgeted deep dive (<=3 tools, <=$0.25) | ✅ PASS | 61.1 | - |

## Disclosure Metrics (P0 Instrumentation)

### Disclosure Level by Model

| Model | Full | Partial | None |
|-------|------|---------|------|
| claude-haiku-4.5 | 0 | 24 | 0 |
| gpt-5-mini | 0 | 23 | 0 |
| gemini-3-flash | 0 | 24 | 0 |
| deepseek-r1 | 0 | 23 | 0 |
| deepseek-v3.2 | 0 | 21 | 0 |
| qwen3-235b | 0 | 24 | 0 |
| minimax-m2.1 | 0 | 24 | 0 |
| mimo-v2-flash-free | 0 | 24 | 0 |

### Disclosure Warnings (Non-Scored)

✅ No disclosure warnings generated.
