# Fully Parallel Evaluation Results

Generated: 2026-01-12T20:16:11.023Z
Total Time: 140.1s
Suite: core
Models: 11
Scenarios: 3 of 3 (limit=10)
Total evaluations: 33

## Summary by Model

| Model | Total | Passed | Failed | Avg Time (s) |
|-------|-------|--------|--------|--------------|
| mimo-v2-flash-free | 3 | 3 | 0 | 102.6 |
| devstral-2-free | 3 | 0 | 3 | 75.6 |
| deepseek-r1-free | 3 | 0 | 3 | 86.4 |
| llama-4-maverick-free | 3 | 0 | 3 | 82.4 |
| llama-4-scout-free | 3 | 0 | 3 | 97.2 |
| glm-4.5-air-free | 3 | 0 | 3 | 53.0 |
| kat-coder-pro-free | 3 | 0 | 3 | 66.2 |
| deepseek-chimera-free | 3 | 0 | 3 | 78.3 |
| grok-4-fast-free | 3 | 0 | 3 | 91.4 |
| venice-dolphin-free | 3 | 0 | 3 | 4.3 |
| nemotron-free | 3 | 0 | 3 | 4.5 |

## Summary by Scenario

| Scenario | Total | Passed | Failed |
|----------|-------|--------|--------|
| Banker vague outreach debrief | 11 | 1 | 10 |
| CTO risk exposure + patch plan | 11 | 1 | 10 |
| Exec vendor evaluation | 11 | 1 | 10 |

## Detailed Results

### mimo-v2-flash-free

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Banker vague outreach debrief | ✅ PASS | 133.6 | - |
| CTO risk exposure + patch plan | ✅ PASS | 116.8 | - |
| Exec vendor evaluation | ✅ PASS | 57.4 | - |

### devstral-2-free

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Banker vague outreach debrief | ❌ FAIL | 126.2 | Missing [DEBRIEF_V1_JSON] block |
| CTO risk exposure + patch plan | ❌ FAIL | 73.6 | Missing [DEBRIEF_V1_JSON] block |
| Exec vendor evaluation | ❌ FAIL | 27.1 | Missing [DEBRIEF_V1_JSON] block |

### deepseek-r1-free

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Banker vague outreach debrief | ❌ FAIL | 80.7 | Missing [DEBRIEF_V1_JSON] block |
| CTO risk exposure + patch plan | ❌ FAIL | 102.2 | Missing [DEBRIEF_V1_JSON] block |
| Exec vendor evaluation | ❌ FAIL | 76.2 | Missing [DEBRIEF_V1_JSON] block |

### llama-4-maverick-free

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Banker vague outreach debrief | ❌ FAIL | 72.3 | Missing [DEBRIEF_V1_JSON] block |
| CTO risk exposure + patch plan | ❌ FAIL | 89.7 | Missing [DEBRIEF_V1_JSON] block |
| Exec vendor evaluation | ❌ FAIL | 85.3 | Missing [DEBRIEF_V1_JSON] block |

### llama-4-scout-free

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Banker vague outreach debrief | ❌ FAIL | 140.1 | Missing [DEBRIEF_V1_JSON] block |
| CTO risk exposure + patch plan | ❌ FAIL | 74.7 | Missing [DEBRIEF_V1_JSON] block |
| Exec vendor evaluation | ❌ FAIL | 76.6 | Missing [DEBRIEF_V1_JSON] block |

### glm-4.5-air-free

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Banker vague outreach debrief | ❌ FAIL | 29.9 | Missing [DEBRIEF_V1_JSON] block |
| CTO risk exposure + patch plan | ❌ FAIL | 70.9 | Missing [DEBRIEF_V1_JSON] block |
| Exec vendor evaluation | ❌ FAIL | 58.2 | Missing [DEBRIEF_V1_JSON] block |

### kat-coder-pro-free

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Banker vague outreach debrief | ❌ FAIL | 45.4 | Missing [DEBRIEF_V1_JSON] block |
| CTO risk exposure + patch plan | ❌ FAIL | 73.0 | Missing [DEBRIEF_V1_JSON] block |
| Exec vendor evaluation | ❌ FAIL | 80.3 | Missing [DEBRIEF_V1_JSON] block |

### deepseek-chimera-free

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Banker vague outreach debrief | ❌ FAIL | 130.6 | Missing [DEBRIEF_V1_JSON] block |
| CTO risk exposure + patch plan | ❌ FAIL | 74.7 | Missing [DEBRIEF_V1_JSON] block |
| Exec vendor evaluation | ❌ FAIL | 29.6 | Missing [DEBRIEF_V1_JSON] block |

### grok-4-fast-free

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Banker vague outreach debrief | ❌ FAIL | 72.9 | Missing [DEBRIEF_V1_JSON] block |
| CTO risk exposure + patch plan | ❌ FAIL | 130.9 | Missing [DEBRIEF_V1_JSON] block |
| Exec vendor evaluation | ❌ FAIL | 70.5 | Missing [DEBRIEF_V1_JSON] block |

### venice-dolphin-free

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Banker vague outreach debrief | ❌ FAIL | 3.8 | Missing [DEBRIEF_V1_JSON] block |
| CTO risk exposure + patch plan | ❌ FAIL | 4.0 | Missing [DEBRIEF_V1_JSON] block |
| Exec vendor evaluation | ❌ FAIL | 5.0 | Missing [DEBRIEF_V1_JSON] block |

### nemotron-free

| Scenario | Status | Time (s) | Failures |
|----------|--------|----------|----------|
| Banker vague outreach debrief | ❌ FAIL | 4.5 | Missing [DEBRIEF_V1_JSON] block |
| CTO risk exposure + patch plan | ❌ FAIL | 4.6 | Missing [DEBRIEF_V1_JSON] block |
| Exec vendor evaluation | ❌ FAIL | 4.5 | Missing [DEBRIEF_V1_JSON] block |

## Disclosure Metrics (P0 Instrumentation)

### Disclosure Level by Model

| Model | Full | Partial | None |
|-------|------|---------|------|
| mimo-v2-flash-free | 0 | 3 | 0 |
| devstral-2-free | 0 | 0 | 3 |
| deepseek-r1-free | 0 | 0 | 3 |
| llama-4-maverick-free | 0 | 0 | 3 |
| llama-4-scout-free | 0 | 0 | 3 |
| glm-4.5-air-free | 0 | 0 | 3 |
| kat-coder-pro-free | 0 | 0 | 3 |
| deepseek-chimera-free | 0 | 0 | 3 |
| grok-4-fast-free | 0 | 0 | 3 |
| venice-dolphin-free | 0 | 0 | 3 |
| nemotron-free | 0 | 0 | 3 |

### Disclosure Warnings (Non-Scored)

✅ No disclosure warnings generated.
