# Full-Stack Eval

Generated: 2026-04-22T18:37:29.741Z
Readiness: not_ready

## Phase Status

| Phase | Status | Duration |
|---|---|---|
| typecheck | passed | 1.5s |
| build | passed | 71.1s |
| capability_compare | passed | 364.7s |
| expanded_feature_capability | passed | 588.9s |
| product_answer_control | failed | 4.9s |
| dogfood_strict | failed | 330.4s |
| notebook_capacity | passed | 202.7s |
| product_history_soak | passed | 21.4s |

## Capability

Overall pass rate: 100.0%
LLM judge average: 9.4/10
p95 latency: 32.6s
Entity correct: 100.0%
Factually accurate: 78.1%
No hallucinations: 71.9%
Actionable: 100.0%

| Model | Pass Rate | Judge Score | Avg Latency |
|---|---|---|---|
| gpt-5.4 | 100.0% | 9.4 | 24.3s |

## Feature Breadth

Expanded pass rate: 9.7%
Expanded model: gpt-5.4 | category: all | total time: 584.2s

| Category | Passed | Total | Pass Rate | Avg Latency |
|---|---|---|---|---|
| calendar | 3 | 3 | 100.0% | 28.4s |
| disclosure | 0 | 4 | 0.0% | 16.3s |
| document | 0 | 3 | 0.0% | 18.4s |
| hybrid | 0 | 4 | 0.0% | 15.8s |
| media | 0 | 3 | 0.0% | 18.5s |
| skills | 0 | 4 | 0.0% | 19.3s |
| spreadsheet | 0 | 3 | 0.0% | 16.6s |
| tools | 0 | 4 | 0.0% | 21.5s |
| web | 0 | 3 | 0.0% | 15.8s |

## Answer Control

Answer-control pass rate: 0.0%
Entity resolution: 0.0%
Artifact decision quality: 0.0%
Ambiguity recovery: 0.0%

| Dimension | Passed | Total | Pass Rate |
|---|---|---|---|
| entity_resolution | 0 | 5 | 0.0% |
| retrieval_relevance | 5 | 5 | 100.0% |
| claim_support | 0 | 5 | 0.0% |
| final_response_quality | 0 | 5 | 0.0% |
| trajectory_quality | 0 | 5 | 0.0% |
| actionability | 0 | 5 | 0.0% |
| artifact_decision_quality | 0 | 5 | 0.0% |
| ambiguity_recovery | 0 | 2 | 0.0% |

## UX / Dogfood

Dogfood score: 93/100 (A)
Real issues: 1

## Capacity

Notebook worst p95: 241ms | worst unexpected error rate: 0.00%
History soak worst p95: 769ms | worst error rate: 0.00%

### Notebook Load Rows

| Scenario | p95 | Error % | Notes |
|---|---|---|---|
| concurrent_insert | 241ms | 0.00% |  |
| sustained_append | 165ms | 0.00% |  |
| paginated_read | 125ms | 0.00% |  |
| multi_tab_edit | 149ms | 0.00% |  |
| multi_tab_conflict | 138ms | 65.00% | expected conflict guards: REVISION_MISMATCH |
| shared_session_insert | 167ms | 0.00% |  |

### Product History Rows

| Scenario | p95 | Error % |
|---|---|---|
| start_session | 283ms | 0.00% |
| complete_session | 491ms | 0.00% |
| session_e2e | 769ms | 0.00% |
| list_sessions | 107ms | 0.00% |
| get_session | 147ms | 0.00% |
| get_session_messages | 136ms | 0.00% |
| list_reports | 108ms | 0.00% |
| get_report | 130ms | 0.00% |
| get_entity_notebook | 129ms | 0.00% |

## Verdict

Capability OK: true
Expanded Feature Coverage OK: false
Answer Control OK: false
Dogfood OK: false
Notebook OK: true
History OK: true
Failing phases: product_answer_control, dogfood_strict

## Artifacts

- [Full-stack JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/full-stack-eval-2026-04-22T18-11-03.json)
- [Capability JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/comprehensive-eval-2026-04-22T18-18-21-metrics.json)
- [Expanded eval JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/expanded-eval-2026-04-22T18-28-10.json)
- [Answer-control JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/product-answer-control-eval-2026-04-22T18-28-15.json)
- [Notebook load JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/.tmp/evals/notebook-load-latest.json)
- [Product history JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/.tmp/evals/product-chat-history-latest.json)