# Full-Stack Eval

Generated: 2026-04-23T08:21:49.541Z
Readiness: not_ready

## Phase Status

| Phase | Status | Duration |
|---|---|---|
| typecheck | skipped | 0ms |
| build | skipped | 0ms |
| capability_compare | skipped | 0ms |
| expanded_feature_capability | skipped | 0ms |
| product_answer_control | skipped | 0ms |
| retention_continuity | passed | 1.8s |
| ultra_long_chat | passed | 5.1s |
| dogfood_strict | skipped | 0ms |
| notebook_capacity | skipped | 0ms |
| product_history_soak | skipped | 0ms |

## Capability

Overall pass rate: 100.0%
LLM judge average: 10.0/10
p95 latency: 171.2s
Entity correct: 100.0%
Factually accurate: 100.0%
No hallucinations: 100.0%
Actionable: 100.0%

| Model | Pass Rate | Judge Score | Avg Latency |
|---|---|---|---|
| kimi-k2.6 | 100.0% | 10.0 | 171.2s |

## Feature Breadth

Expanded pass rate: 100.0%
Expanded model: kimi-k2.6 | category: all | total time: 154.2s

| Category | Passed | Total | Pass Rate | Avg Latency |
|---|---|---|---|---|
| tools | 1 | 1 | 100.0% | 154.2s |

## Answer Control

Answer-control pass rate: 100.0%
Entity resolution: 100.0%
Artifact decision quality: 100.0%
Ambiguity recovery: 100.0%

| Dimension | Passed | Total | Pass Rate |
|---|---|---|---|
| entity_resolution | 5 | 5 | 100.0% |
| retrieval_relevance | 5 | 5 | 100.0% |
| claim_support | 5 | 5 | 100.0% |
| final_response_quality | 5 | 5 | 100.0% |
| trajectory_quality | 5 | 5 | 100.0% |
| actionability | 5 | 5 | 100.0% |
| artifact_decision_quality | 5 | 5 | 100.0% |
| ambiguity_recovery | 2 | 2 | 100.0% |

## Continuation

Retention pass rate: 100.0%
Retention summary: Passed: 4/4 scenarios
Ultra-long passed: true
Ultra-long progressive score: 80
Ultra-long real-path score: 102
Ultra-long savings vs kitchen sink: 50%

### Retention Risk Areas

- none

## UX / Dogfood

Dogfood score: 100/100 (A)
Real issues: 0

## Capacity

Notebook worst p95: 172ms | worst unexpected error rate: 0.00%
History soak worst p95: 495ms | worst error rate: 0.00%

### Notebook Load Rows

| Scenario | p95 | Error % | Notes |
|---|---|---|---|
| concurrent_insert | 148ms | 0.00% |  |
| sustained_append | 172ms | 0.00% |  |
| paginated_read | 127ms | 0.00% |  |
| multi_tab_edit | 143ms | 0.00% |  |
| multi_tab_conflict | 147ms | 64.65% | expected conflict guards: REVISION_MISMATCH |
| shared_session_insert | 146ms | 0.00% |  |

### Product History Rows

| Scenario | p95 | Error % |
|---|---|---|
| start_session | 223ms | 0.00% |
| complete_session | 263ms | 0.00% |
| session_e2e | 495ms | 0.00% |
| list_sessions | 105ms | 0.00% |
| get_session | 166ms | 0.00% |
| get_session_messages | 110ms | 0.00% |
| list_reports | 109ms | 0.00% |
| get_report | 109ms | 0.00% |
| get_entity_notebook | 120ms | 0.00% |

## Verdict

Capability Next Gate OK: false
Capability Production Gate OK: false
Expanded Feature Coverage Next Gate OK: true
Expanded Feature Coverage Production Gate OK: true
Answer Control Next Gate OK: true
Answer Control Production Gate OK: true
Dogfood Next Gate OK: true
Dogfood Production Gate OK: true
Notebook Next Gate OK: true
Notebook Production Gate OK: true
History Next Gate OK: true
History Production Gate OK: true
Retention Next Gate OK: true
Retention Production Gate OK: true
Ultra-Long Next Gate OK: true
Ultra-Long Production Gate OK: true
Failing phases: none

## Artifacts

- [Full-stack JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/full-stack-eval-2026-04-23T08-21-42.json)
- [Capability JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/comprehensive-eval-2026-04-23T07-05-42-metrics.json)
- [Expanded eval JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/expanded-eval-2026-04-23T07-02-18.json)
- [Answer-control JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/product-answer-control-eval-2026-04-23T06-43-12.json)
- [Retention JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/retention-eval-latest.json)
- [Ultra-long JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/docs/architecture/benchmarks/ultra-long-chat-eval-latest.json)
- [Notebook load JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/.tmp/evals/notebook-load-latest.json)
- [Product history JSON](D:/VSCode Projects/cafecorner_nodebench/nodebench_ai4/nodebench-ai/.tmp/evals/product-chat-history-latest.json)