# Expanded Evaluation Results

Generated: 2026-04-22T18:28:10.093Z
Model: gpt-5.4
Category: all
Total Time: 584.2s

## Summary

| Metric | Value |
|--------|-------|
| Total Scenarios | 31 |
| Passed | 3 |
| Failed | 28 |
| Pass Rate | 9.7% |

## Results by Category

### Calendar

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Calendar: Schedule DISCO review meeting | ✅ PASS | 24.3s | searchAvailableSkills, lookupGroundTruthEntity, searchAvailableSkills, lookupGroundTruthEntity, createEvent | - |
| Calendar: Schedule Salesforce demo | ✅ PASS | 36.0s | searchAvailableSkills, lookupGroundTruthEntity, linkupSearch, searchAvailableSkills, lookupGroundTruthEntity, getSystemDateTime, linkupSearch, listEvents, getUserContext, createEvent | - |
| Calendar: Schedule Ambros follow-up | ✅ PASS | 24.9s | searchAvailableSkills, lookupGroundTruthEntity, searchAvailableSkills, lookupGroundTruthEntity, getSystemDateTime, createEvent | - |

### Spreadsheet

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Spreadsheet: DISCO funding metrics | ❌ FAIL | 19.5s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Spreadsheet: Ambros deal pipeline | ❌ FAIL | 15.4s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Spreadsheet: Gemini pricing comparison | ❌ FAIL | 15.0s | none | Missing required tool call: lookupGroundTruthEntity; Output  |

### Document

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Document: DISCO investment memo | ❌ FAIL | 21.4s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Document: QuickJS security assessment | ❌ FAIL | 16.8s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Document: GenomiQ due diligence | ❌ FAIL | 16.9s | none | Missing required tool call: lookupGroundTruthEntity; Output  |

### Web

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Web: DISCO news search | ❌ FAIL | 15.6s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Web: Salesforce Agentforce deep dive | ❌ FAIL | 16.0s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Web: DISCO multi-source verification | ❌ FAIL | 15.7s | none | Missing required tool call: lookupGroundTruthEntity; Output  |

### Media

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Media: DISCO UI card design | ❌ FAIL | 16.8s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Media: SoundCloud incident timeline | ❌ FAIL | 16.5s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Media: VaultPay sales materials | ❌ FAIL | 22.1s | none | Missing required tool call: lookupGroundTruthEntity; Output  |

### Skills

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Skills: Banker skill discovery | ❌ FAIL | 15.9s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |
| Skills: VC thesis skill | ❌ FAIL | 24.9s | none | Output missing expected content: "OpenAutoGLM"; Missing [DEB |
| Skills: Quant signal skill | ❌ FAIL | 15.0s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |
| Skills: Digest persona skill | ❌ FAIL | 21.3s | none | Output missing expected content: "digest"; Output missing ex |

### Tools

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Tools: Basic tool discovery | ❌ FAIL | 39.1s | searchAvailableSkills, lookupGroundTruthEntity | Missing required tool call: searchAvailableTools |
| Tools: Describe tool schemas | ❌ FAIL | 17.4s | none | Missing required tool call: describeTools; Output missing ex |
| Tools: Gateway invocation | ❌ FAIL | 14.6s | none | Missing required tool call: invokeTool; Output missing expec |
| Tools: Safe tier tool | ❌ FAIL | 14.7s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |

### Hybrid

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Hybrid: Memory-first retrieval | ❌ FAIL | 16.9s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |
| Hybrid: Fusion search ranking | ❌ FAIL | 14.0s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |
| Hybrid: Cross-source aggregation | ❌ FAIL | 17.9s | none | Output missing expected content: "DISCO"; Output missing exp |
| Hybrid: Stale data detection | ❌ FAIL | 14.3s | none | Output missing expected content: "ClearSpace"; Missing [DEBR |

### Disclosure

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Disclosure: P0 telemetry events | ❌ FAIL | 16.5s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |
| Disclosure: P1 persona classification | ❌ FAIL | 16.4s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |
| Disclosure: P2 skill allowlist | ❌ FAIL | 17.2s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |
| Disclosure: P3 full disclosure trace | ❌ FAIL | 15.2s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |
