# Full-Stack Eval

Generated: 2026-04-21T21:39:13.423Z
Readiness: not_ready

## Phase Status

| Phase | Status | Duration |
|---|---|---|
| typecheck | skipped | 0ms |
| build | skipped | 0ms |
| capability_compare | passed | 1782.6s |
| expanded_feature_capability | passed | 1509.5s |
| dogfood_strict | passed | 970.9s |
| notebook_capacity | passed | 199.2s |
| product_history_soak | passed | 20.2s |

## Capability

Overall pass rate: 57.3%
LLM judge average: 5.3/10
p95 latency: 81.8s

| Model | Pass Rate | Judge Score | Avg Latency |
|---|---|---|---|
| claude-haiku-4.5 | 84.4% | 5.1 | 57.9s |
| gpt-5-mini | 87.5% | 5.5 | 40.3s |
| gemini-3-flash | 0.0% | 0.0 | 10.1s |

## Feature Breadth

Expanded pass rate: 71.0%
Expanded model: gpt-5-mini | category: all | total time: 1506.5s

| Category | Passed | Total | Pass Rate | Avg Latency |
|---|---|---|---|---|
| calendar | 3 | 3 | 100.0% | 37.1s |
| disclosure | 3 | 4 | 75.0% | 58.3s |
| document | 2 | 3 | 66.7% | 48.1s |
| hybrid | 2 | 4 | 50.0% | 58.9s |
| media | 1 | 3 | 33.3% | 57.3s |
| skills | 2 | 4 | 50.0% | 44.8s |
| spreadsheet | 3 | 3 | 100.0% | 35.4s |
| tools | 3 | 4 | 75.0% | 39.9s |
| web | 3 | 3 | 100.0% | 55.2s |

## UX / Dogfood

Dogfood score: 97/100 (A)
Real issues: 4

## Capacity

Notebook worst p95: 168ms | worst unexpected error rate: 0.00%
History soak worst p95: 646ms | worst error rate: 0.00%

### Notebook Load Rows

| Scenario | p95 | Error % | Notes |
|---|---|---|---|
| concurrent_insert | 147ms | 0.00% |  |
| sustained_append | 168ms | 0.00% |  |
| paginated_read | 130ms | 0.00% |  |
| multi_tab_edit | 147ms | 0.00% |  |
| multi_tab_conflict | 137ms | 55.61% | expected conflict guards: REVISION_MISMATCH |
| shared_session_insert | 151ms | 0.00% |  |

### Product History Rows

| Scenario | p95 | Error % |
|---|---|---|
| start_session | 187ms | 0.00% |
| complete_session | 531ms | 0.00% |
| session_e2e | 646ms | 0.00% |
| list_sessions | 98ms | 0.00% |
| get_session | 111ms | 0.00% |
| get_session_messages | 108ms | 0.00% |
| list_reports | 104ms | 0.00% |
| get_report | 103ms | 0.00% |
| get_entity_notebook | 125ms | 0.00% |

## Verdict

Capability OK: false
Expanded Feature Coverage OK: true
Dogfood OK: false
Notebook OK: true
History OK: true
Failing phases: none

## Artifacts

- [Full-stack JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/full-stack-eval-2026-04-21T20-24-30.json)
- [Capability JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/comprehensive-eval-2026-04-21T20-54-13-metrics.json)
- [Expanded eval JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/expanded-eval-2026-04-21T21-19-23.json)
- [Notebook load JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/.tmp/evals/notebook-load-latest.json)
- [Product history JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/.tmp/evals/product-chat-history-latest.json)