# M24.2 — Producer audit (hot-path)

## Context

M24.1 shipped the lifecycle kernel + ``ops_state`` projection.
The kernel classifies items from ``audit_events`` rows that
producers are *supposed* to emit.  As ``docs/operational-lifecycle.md``
warned in the honest-zero section, three different upstream causes
collapse into "0 evidence":

* the producer didn't run,
* it ran with no rows,
* it ran but **doesn't emit the audit row we're counting**.

Cause 3 is what M24.2 fixes.  Without it, the kernel mis-classifies
items as Prepared / Projected whenever a producer that *should*
write a row silently skips that write.  Operator never sees the bug
because the count looks like "everyone else is just quiet today".

A code-spelunking pass during M24.1's design phase already turned
up two missing producers the kernel actively depends on:

* **``candidates_upserted``** — registered in
  ``event_evidence_registry`` (line 110), referenced by the kernel
  (``ops_lifecycle.py:428``) and by ``truth_api`` (lines 4454,
  4668) and ``_truth_helpers`` (line 150), **never emitted by any
  producer**.  Result: every source with
  ``evergreen_extraction_complete`` lands at sub-state ``Prepared``
  on the live vault.
* **``promote_concept``** — registered in the registry (line 126),
  referenced by the kernel as one of two events that flip an item
  to Accepted (``ops_lifecycle.py:158, 415``), **never emitted**.
  Result: items promoted via operator action (the ``promote``
  command) never land in Accepted; only auto-promoted ones do.

There is also one new event_type the M24.1 plan explicitly called
for that we haven't shipped yet:

* **``absorb_pending_upsert``** — fired by the evergreen extractor
  when extraction finishes but the candidate row hasn't landed in
  ``knowledge.db`` yet.  The kernel uses this to disambiguate
  Prepared from a normal in-flight extraction.  Without it, the
  Prepared sub-state is over-broad.

## Scope (locks)

This milestone follows the M24/M25-wide locks captured in
``docs/plans/2026-05-14-m24-lifecycle-contract-layer.md``:

* **Hot-path only.**  Wire the three event_types above and audit
  *only* the producers the kernel reads from for state classification:
  ``auto_article_processor``, ``clippings_processor``,
  ``auto_github_processor``, ``auto_evergreen_extractor``,
  ``absorb_router``, plus whichever module owns operator-promote
  (likely ``commands/promote.py`` or ``promote_candidates.py``).
  ~7 producers, not the full 20+.
* **No surface changes.**  No new cards, no new lifecycle states.
  M25 still owns surface design.
* **Honest-zero stays in place.**  Even after M24.2 wires the
  missing producers, surfaces continue to read the ambiguity
  helper — the producer audit is *diagnostic*, not a guarantee
  that every zero is now diagnosable.
* **No producer rewrites.**  M24.2 inserts the missing emit calls
  at the right point in each producer's existing flow.  It does
  not refactor producers onto a unified emitter, doesn't add
  retries, and doesn't change behaviour beyond emitting one new
  audit row per existing decision point.

## Stages

### Stage 2.A — Producer contracts module

**Location:** ``src/ovp_pipeline/producer_audit.py``.

Declarative registry that says, for each hot-path producer:

* its name,
* the file path that owns the producer (for diagnostic output),
* the set of audit event_types it MUST emit,
* the set of audit event_types it MAY emit (governed by config /
  branches the producer takes).

Public surface:

```python
@dataclass(frozen=True)
class ProducerContract:
    producer: str
    module: str
    must_emit: tuple[str, ...]
    may_emit: tuple[str, ...] = ()
    description: str = ""

CONTRACTS: tuple[ProducerContract, ...]

def producer_for_event_type(event_type: str) -> ProducerContract | None
def audit_against_log(
    conn: sqlite3.Connection,
    *,
    window_start: str | None = None,
) -> ProducerAuditReport
```

``audit_against_log`` returns:

* per producer: whether each ``must_emit`` event appeared in
  ``audit_events`` since ``window_start`` (default 7 days),
* event_types in ``audit_events`` that don't map to any declared
  producer (instrumentation drift — a producer that started
  emitting something we didn't promise).

Every event_type the kernel reads must be on at least one
producer's ``must_emit`` list.  A test enforces this — adding a
new state-affecting event_type to ``event_evidence_registry``
without picking up a producer here fails CI loudly.

### Stage 2.B — Wire missing producers

Three emit sites land in this milestone:

1. **``candidates_upserted``** — emit in
   ``auto_evergreen_extractor.process_file`` (or whichever helper
   actually writes candidate rows to ``knowledge.db``) after the
   insert succeeds.  Payload: ``{source, candidates: int}``.

2. **``promote_concept``** — emit in ``commands/promote.py`` (or
   ``promote_candidates.py``, whichever wraps the operator-promote
   path) after the promote completes.  Payload:
   ``{concept, source, path}``.

3. **``absorb_pending_upsert``** — emit in
   ``auto_evergreen_extractor.process_file`` BETWEEN
   ``evergreen_extraction_complete`` and ``candidates_upserted``,
   i.e. right after extraction returns but before the candidate
   write.  Payload: ``{source, expected_candidates: int}``.  When
   the upsert that follows succeeds, the pair surfaces a clean
   transition; when it doesn't, the pending row stays in the log
   and the kernel sees Prepared sub-state with a real anchor.

All three use the existing ``event_emitter.emit()`` API (no new
infrastructure).

### Stage 2.C — Registry: ``absorb_pending_upsert``

Add one row to ``_REGISTRY`` in ``event_evidence_registry.py``:

```python
EventEvidence(
    "absorb_pending_upsert", "absorb",
    user_visible=False,
    description=(
        "Extraction finished but the candidate upsert hasn't "
        "been observed yet — used by the kernel to detect "
        "Prepared sub-state precisely."
    ),
),
```

The other two (``candidates_upserted``, ``promote_concept``) are
already registered; M24.2 only adds the missing entry.

### Stage 2.D — CLI surface

**``ovp-producer-audit``** — one entry point.  Two modes:

* default (``--report``) — print a table: producer, expected
  events, last-seen timestamps, missing events flagged.
* ``--json`` — same data as a JSON payload, suitable for
  ``/ops/digest-health`` consumption in a later milestone.

No write side.  The CLI is read-only over ``knowledge.db``.

### Stage 2.E — Tests

* ``tests/test_producer_audit.py`` — fixture vault with a
  curated set of audit rows; assertions on which producers the
  audit flags as missing.
* ``tests/test_producer_audit.py::test_every_kernel_event_type_has_a_producer``
  — enforces the cross-module invariant.
* Extend ``tests/test_event_evidence_registry.py`` with the new
  ``absorb_pending_upsert`` row.
* Extend ``tests/test_ops_lifecycle.py`` with a Prepared case that
  consumes ``absorb_pending_upsert`` instead of just the absence
  of ``candidates_upserted``.
* Add a small smoke test exercising the three new producer emit
  sites end-to-end (process a fake source through
  ``auto_evergreen_extractor`` and confirm the three rows land).

## Non-goals

* No unified emit API.  Producers keep using whatever they use
  (``PipelineLogger`` or ``event_emitter.emit``); M24.2 only adds
  the missing calls.
* No "instrumentation health card" on Maintainer surfaces — that's
  M25's call.
* No retry / queueing if an emit fails.  ``event_emitter.emit``
  already writes atomically; a failure surfaces as the row simply
  not existing, which is exactly what the audit report flags.

## Acceptance checklist

* [ ] ``src/ovp_pipeline/producer_audit.py`` + tests merged.
* [ ] ``candidates_upserted`` emitted by the absorb writer.
* [ ] ``promote_concept`` emitted by the operator-promote path.
* [ ] ``absorb_pending_upsert`` emitted + registered.
* [ ] ``ovp-producer-audit`` CLI lands; ``--report`` against the
      operator vault shows zero missing on the hot-path
      producers.
* [ ] Kernel test exercising the Prepared sub-state with the new
      ``absorb_pending_upsert`` event still passes.
* [ ] Full ``pytest`` stays green (modulo the two pre-existing
      architecture-fitness violations).

---

*Plan author: Claude (M24.2).  Locks honored from
docs/plans/2026-05-14-m24-lifecycle-contract-layer.md.*
