# Expanded Evaluation Results

Generated: 2026-04-23T03:44:36.935Z
Model: kimi-k2.6
Category: all
Total Time: 3348.9s

## Summary

| Metric | Value |
|--------|-------|
| Total Scenarios | 31 |
| Passed | 8 |
| Failed | 23 |
| Pass Rate | 25.8% |

## Results by Category

### Calendar

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Calendar: Schedule DISCO review meeting | ❌ FAIL | 126.3s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Calendar: Schedule Salesforce demo | ✅ PASS | 128.5s | searchAvailableSkills, lookupGroundTruthEntity, linkupSearch, createEvent | - |
| Calendar: Schedule Ambros follow-up | ❌ FAIL | 102.0s | none | Missing required tool call: lookupGroundTruthEntity; Output  |

### Spreadsheet

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Spreadsheet: DISCO funding metrics | ✅ PASS | 64.5s | searchAvailableSkills, lookupGroundTruthEntity, searchAvailableSkills, lookupGroundTruthEntity, createSpreadsheet | - |
| Spreadsheet: Ambros deal pipeline | ✅ PASS | 87.0s | searchAvailableSkills, lookupGroundTruthEntity, searchAvailableSkills, lookupGroundTruthEntity, listSpreadsheets, createSpreadsheet | - |
| Spreadsheet: Gemini pricing comparison | ❌ FAIL | 109.3s | none | Missing required tool call: lookupGroundTruthEntity; Output  |

### Document

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Document: DISCO investment memo | ❌ FAIL | 116.6s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Document: QuickJS security assessment | ❌ FAIL | 113.7s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Document: GenomiQ due diligence | ❌ FAIL | 112.4s | none | Missing required tool call: lookupGroundTruthEntity; Output  |

### Web

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Web: DISCO news search | ❌ FAIL | 113.7s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Web: Salesforce Agentforce deep dive | ❌ FAIL | 114.7s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Web: DISCO multi-source verification | ✅ PASS | 99.0s | searchAvailableSkills, lookupGroundTruthEntity, searchAvailableSkills, lookupGroundTruthEntity, linkupSearch, linkupSearch | - |

### Media

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Media: DISCO UI card design | ❌ FAIL | 116.9s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Media: SoundCloud incident timeline | ❌ FAIL | 114.9s | none | Missing required tool call: lookupGroundTruthEntity; Output  |
| Media: VaultPay sales materials | ❌ FAIL | 112.4s | none | Missing required tool call: lookupGroundTruthEntity; Output  |

### Skills

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Skills: Banker skill discovery | ✅ PASS | 53.6s | searchAvailableSkills, lookupGroundTruthEntity, searchAvailableSkills, lookupGroundTruthEntity, describeSkill, describeSkill | - |
| Skills: VC thesis skill | ❌ FAIL | 118.2s | none | Output missing expected content: "OpenAutoGLM"; Missing [DEB |
| Skills: Quant signal skill | ✅ PASS | 75.8s | searchAvailableSkills, lookupGroundTruthEntity, searchAvailableSkills, initScratchpad, describeSkill, lookupGroundTruthEntity, getBankerGradeEntityInsights, evaluateEntityForPersona, searchFundingEvents, linkupSearch | - |
| Skills: Digest persona skill | ❌ FAIL | 117.9s | none | Output missing expected content: "digest"; Output missing ex |

### Tools

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Tools: Basic tool discovery | ❌ FAIL | 109.8s | none | Missing required tool call: searchAvailableTools; Output mis |
| Tools: Describe tool schemas | ❌ FAIL | 116.7s | none | Missing required tool call: describeTools; Output missing ex |
| Tools: Gateway invocation | ❌ FAIL | 112.9s | none | Missing required tool call: invokeTool; Output missing expec |
| Tools: Safe tier tool | ❌ FAIL | 108.9s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |

### Hybrid

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Hybrid: Memory-first retrieval | ✅ PASS | 54.6s | searchAvailableSkills, lookupGroundTruthEntity, searchAvailableSkills, queryMemory, lookupGroundTruthEntity | - |
| Hybrid: Fusion search ranking | ❌ FAIL | 119.4s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |
| Hybrid: Cross-source aggregation | ❌ FAIL | 131.3s | none | Output missing expected content: "DISCO"; Output missing exp |
| Hybrid: Stale data detection | ❌ FAIL | 110.8s | none | Output missing expected content: "ClearSpace"; Missing [DEBR |

### Disclosure

| Scenario | Status | Time | Tools Called | Issues |
|----------|--------|------|--------------|--------|
| Disclosure: P0 telemetry events | ❌ FAIL | 121.3s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |
| Disclosure: P1 persona classification | ❌ FAIL | 161.7s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |
| Disclosure: P2 skill allowlist | ✅ PASS | 91.1s | searchAvailableSkills, lookupGroundTruthEntity, searchAvailableSkills | - |
| Disclosure: P3 full disclosure trace | ❌ FAIL | 112.9s | none | Output missing expected content: "DISCO"; Missing [DEBRIEF_V |
