# OpenClerk Agent Eval Status

This file is a status pointer, not a raw eval report.

The previous contents recorded an older partial production run where
`production` passed 1/17 selected scenarios. That run is superseded and should
not be used as current OpenClerk AgentOps status.

## Current Full Production Gate

Use `docs/evals/results/ockp-agentops-production.md` as the current full
production gate artifact.

- Variant: `production`
- Gate: passing
- Scenario coverage: 30/30 production scenarios passed
- Recommendation: use the AgentOps runner for routine OpenClerk operations
- Raw logs: `<run-root>/<variant>/<scenario>/turn-N/events.jsonl`

That report is the current release-gate evidence for routine OpenClerk
knowledge workflows through the installed `openclerk` runner.

## Mandatory Pre-Release Dogfood

Use `docs/evals/results/ockp-repo-docs-dogfood.md` as mandatory pre-release
dogfood evidence. This targeted lane is separate from the full production
gate, but release prep must refresh it and require every selected repo-docs
dogfood scenario to pass before tagging.

## Later Targeted POC Evidence

The following later reports are targeted POC evidence. Their production gates
may be false because they intentionally selected only a subset of scenarios;
use their selected scenario results and decision notes as POC evidence, not as
full release-gate replacements.

| Area | Report | Status |
| --- | --- | --- |
| Convention-first layout inspection | `docs/evals/results/ockp-layout-configuration.md` | `configured-layout-explain` and `invalid-layout-visible` completed |
| Synthesis compiler pressure | `docs/evals/results/ockp-synthesis-compiler-pressure.md` | Selected synthesis pressure and contract scenarios completed |
| Source URL ingestion POC | `docs/evals/results/ockp-source-url-ingestion-poc.md` | `ingest_source_url` promoted for PDF source URL ingestion |
| Source URL update mode | `docs/evals/results/ockp-source-url-update-mode.md` | Targeted AgentOps coverage passed duplicate create rejection, same-SHA no-op, changed-PDF stale projection visibility, and path-hint conflict behavior |
| Decision records POC | `docs/evals/results/ockp-decision-records-poc.md` | Selected decision lookup and supersession scenarios completed |
| Decision records hardening | `docs/evals/results/ockp-decision-records-hardening.md` | Real ADR migration and decision projection hardening scenarios completed |
| Source-sensitive audit | `docs/evals/results/ockp-source-sensitive-audit-poc.md` | Stale repair and unresolved conflict scenarios completed |
| Graph semantics reference | `docs/evals/results/ockp-graph-semantics-reference-poc.md` | Graph semantic-label pressure scenario completed as reference evidence |
| Memory/router reference | `docs/evals/results/ockp-memory-router-reference-poc.md` | Memory and router pressure scenario completed as reference evidence |
| Memory/router revisit | `docs/evals/results/ockp-memory-router-revisit-pressure.md` | Targeted revisit completed natural and scripted rows with `none` classifications; keep as reference pressure with no memory API, remember/recall action, or autonomous router surface promoted |
| Real-vault dogfooding | `docs/evals/results/ockp-real-vault-agentops-trial.md` | Sanitized aggregate evidence only; raw logs, paths, titles, snippets, citations, document IDs, and raw JSON remain local-only |
| Repo-docs dogfooding | `docs/evals/results/ockp-repo-docs-dogfood.md` | Mandatory pre-release targeted lane over committed public markdown covering retrieval, synthesis maintenance, decision records, release readiness, tag filters, memory-router recall reports, and release synthesis freshness |
| Populated-vault synthetic pressure | `docs/evals/results/ockp-populated-vault-targeted.md`; follow-up `docs/evals/results/ockp-populated-vault-guidance-hardening.md` | Targeted run completed freshness/conflict and synthesis-update pressure; focused follow-up resolved the heterogeneous polluted-evidence guidance failure, with no runner capability gap or product/API promotion |
| Document-this intake pressure | `docs/evals/results/ockp-document-this-intake-pressure.md` | Targeted document-this intake lane covers missing-field clarification, explicit create/override, duplicate candidates, existing updates, and synthesis freshness |
| Document artifact candidate generation | `docs/evals/results/ockp-document-artifact-candidate-generation.md`; follow-up `docs/evals/results/ockp-document-artifact-candidate-ergonomics.md` | Quality lane and refreshed ergonomics scorecard support the existing propose-before-create skill policy; no runner/schema/storage/API/direct-create behavior promoted |
| Duplicate-candidate capture | `docs/evals/results/ockp-capture-duplicate-candidate-update.md`; follow-up `docs/evals/results/ockp-capture-duplicate-candidate-guidance-hardening.md` | Original lane promoted skill-policy clarification; focused hardening made the natural row pass with path-filtered search/list/get evidence while preserving no-write and approval boundaries |
| Low-risk capture | `docs/evals/results/ockp-capture-low-risk-ceremony.md`; follow-up `docs/evals/results/ockp-capture-low-risk-guidance-hardening.md` | Original ceremony lane exposed an ergonomics gap in the natural row; focused guidance hardening made natural, scripted, duplicate-check, and no-bypass controls pass while keeping the surface to skill policy over existing `openclerk document` and `openclerk retrieval` JSON actions |
| Compile synthesis workflow action | `docs/evals/results/ockp-compile-synthesis-workflow-action.md` | Candidate comparison accepted compact handoff and route-clarity shape; refreshed natural row passes safety, capability, and `workflow_action_acceptable` UX |
| Source audit workflow action | `docs/evals/results/ockp-source-audit-workflow-action.md` | Candidate C accepted: promote narrow `source_audit_report`; safety/capability pass, broad contradiction-engine claims remain rejected, and refreshed natural-row UX is `workflow_action_acceptable` |
| Evidence bundle workflow action | `docs/evals/results/ockp-evidence-bundle-workflow-action.md` | Candidate C accepted: promote read-only `evidence_bundle_report`; safety/capability pass and refreshed natural-row UX is `workflow_action_acceptable` |

## Interpretation

OpenClerk is currently proven for the v1 AgentOps runner slice: canonical docs,
source-linked synthesis, promoted records, provenance events, projection
freshness, and final-answer-only rejection gates.

The later targeted POCs do not promote Mem0 or a memory API, remember/recall
actions, an autonomous router, a semantic graph truth layer, a broad
contradiction engine, embeddings/vector DB, browser acquisition, direct
local-file intake, or lower-level storage access. The only new public runner
actions promoted by the workflow-action reports are `compile_synthesis`,
`source_audit_report`, and read-only `evidence_bundle_report`.
