# Zaxy 2.0 Beta.2 Metacognitive and Procedural Hardening Implementation Plan

> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

Goal: Add production-ready metacognitive state tracking and first-class procedural planning diagnostics without turning uncertainty, confidence, or procedures into authoritative facts.

Architecture: Eventloom remains the source of truth. Beta.2 adds strict metacognition event contracts, replay-derived query surfaces, checkout diagnostics, and procedural planning buckets that reuse existing Skill Memory and consolidation projections. All generated metacognitive/procedural outputs are non-authoritative, cited, observable, and excluded from claim-support scoring unless a separate authority path has accepted them.

Tech Stack: Python 3.11+, Eventloom JSONL, existing `MemoryFabric`, Memory Checkout, Skill Memory, consolidation candidates, Typer CLI, MCP Python SDK, pytest, ruff.

---

## Scope Boundary

Included:

- Track known unknowns through `metacognition.unknown.recorded`.
- Track conflicting evidence clusters through `metacognition.conflict.clustered`.
- Track append-only confidence trajectories through `metacognition.confidence.assessed`.
- Track re-verification needs through `metacognition.reverify.requested`.
- Add `MemoryFabric` query surfaces for known unknowns, conflict clusters, confidence trajectories, and re-verification needs.
- Add optional confidence-assessment recording to `get_claim_confidence` without self-reinforcing prior metacognition or primitive trace events.
- Promote procedural memory into explicit applicable/diagnostic/excluded planning buckets.
- Preserve rollback and contradiction diagnostics for procedural memory.
- Extend `memory_skill` to persist `failure_modes`, `rollback`, and `contradiction_reason`.
- Add CLI/MCP surfaces, docs, generated site pages, and internal beta.2 guardrail checks.

Excluded:

- Autonomous belief revision.
- Silent authority promotion.
- Learned confidence models.
- Benchmark-specific retrieval or scoring logic.
- Cross-project global procedure sharing.

## File Structure

Create:

- `src/zaxy/metacognition.py` Strict contracts, builders, deterministic IDs, replay summaries, and eligibility helpers for metacognitive state.

- `tests/test_metacognition.py` Unit tests for builders, validation, replay summaries, and authority boundaries.

Modify:

- `src/zaxy/core.py` Add `MemoryFabric` metacognition methods, confidence trajectory recording, reverify queries, and procedural planning buckets.

- `src/zaxy/extract.py` Add typed extraction for metacognition events and preserve non-authoritative status.

- `src/zaxy/checkout.py` Add metacognition and procedural planning diagnostics/guidance.

- `src/zaxy/reasoning_benchmark.py` Add beta.2 guardrail scoring for metacognition and procedural planning contracts.

- `src/zaxy/__main__.py` Add `zaxy memory reasoning` commands for uncertainty/reverification/procedural planning.

- `src/zaxy/mcp_server.py` and `docs/examples/mcp-tool-contract.json` Add MCP tools and contract snapshot updates.

- `docs/mcp.md`, `docs/graph-schema.md`, `docs/benchmarks.md`, `docs/agent-events.md`, generated `site/docs/*.html` Document beta.2 boundaries and surfaces.

---

### Task 1: Add Metacognition Contracts

Files:

- Create: `src/zaxy/metacognition.py`
- Test: `tests/test_metacognition.py`

- [ ] Step 1: Write failing contract tests

Add tests proving:

    from zaxy.metacognition import (
        build_confidence_assessment_event,
        build_conflict_cluster_event,
        build_known_unknown_event,
        build_reverify_request_event,
        summarize_metacognition_events,
    )

    SOURCE = [{"seq": 7, "hash": "a" * 64}]

    def test_known_unknown_event_is_open_cited_and_non_authoritative() -> None:
        event = build_known_unknown_event(
            actor="agent",
            session_id="agent-1",
            question="Which projection backend caused the latency spike?",
            reason="Checkout had conflicting backend evidence.",
            source_events=SOURCE,
            claim_key="projection-latency-cause",
            gap_type="conflicting_evidence",
            reverify_query="latest cited projection latency cause",
        )

        assert event["event_type"] == "metacognition.unknown.recorded"
        assert event["thread"] == "agent-1"
        assert event["payload"]["status"] == "open"
        assert event["payload"]["authority_status"] == "non_authoritative"
        assert event["payload"]["source_events"] == SOURCE

    def test_confidence_assessment_event_tracks_append_only_point() -> None:
        event = build_confidence_assessment_event(
            actor="zaxy-reasoning",
            session_id="agent-1",
            claim="Projection stale caused failure",
            confidence=0.42,
            support_count=1,
            conflict_count=2,
            evidence=[
                {"citation": "eventloom://agent-1/events/7#aaaaaaaaaaaa", "stance": "support"},
                {"citation": "eventloom://agent-1/events/8#bbbbbbbbbbbb", "stance": "conflict"},
            ],
            method="deterministic_token_overlap_v1",
            requires_reverify=True,
        )

        assert event["event_type"] == "metacognition.confidence.assessed"
        assert event["payload"]["confidence"] == 0.42
        assert event["payload"]["requires_reverify"] is True
        assert event["payload"]["authority_status"] == "non_authoritative"

    def test_conflict_cluster_event_preserves_support_and_conflict_sources() -> None:
        event = build_conflict_cluster_event(
            actor="zaxy-reasoning",
            session_id="agent-1",
            claim_key="projection-latency-cause",
            claim="Projection stale caused failure",
            supporting_source_events=[{"seq": 7, "hash": "a" * 64}],
            conflicting_source_events=[{"seq": 8, "hash": "b" * 64}],
            confidence=0.5,
            reason="Support and conflict evidence both present.",
        )

        assert event["event_type"] == "metacognition.conflict.clustered"
        assert event["payload"]["resolution_status"] == "unresolved"
        assert event["payload"]["authority_status"] == "non_authoritative"

    def test_reverify_request_event_is_open_and_cited() -> None:
        event = build_reverify_request_event(
            actor="zaxy-reasoning",
            session_id="agent-1",
            query="Re-check cited projection latency cause",
            reason="Low confidence and conflict count above zero.",
            source_events=SOURCE,
            priority="high",
            claim_key="projection-latency-cause",
        )

        assert event["event_type"] == "metacognition.reverify.requested"
        assert event["payload"]["status"] == "open"
        assert event["payload"]["priority"] == "high"

    def test_summarize_metacognition_events_returns_open_counts() -> None:
        events = [
            build_known_unknown_event(
                actor="agent",
                session_id="agent-1",
                question="What changed?",
                reason="No cited answer.",
                source_events=SOURCE,
                claim_key="change",
                gap_type="missing_evidence",
                reverify_query="what changed",
            ),
            build_reverify_request_event(
                actor="agent",
                session_id="agent-1",
                query="what changed",
                reason="missing evidence",
                source_events=SOURCE,
                priority="normal",
                claim_key="change",
            ),
        ]

        summary = summarize_metacognition_events(events)

        assert summary["unknown_count"] == 1
        assert summary["open_unknown_count"] == 1
        assert summary["reverify_needed_count"] == 1

- [ ] Step 2: Run tests to verify failure

Run:

    pytest tests/test_metacognition.py --no-cov -q

Expected: fail because `zaxy.metacognition` does not exist.

- [ ] Step 3: Implement contracts

Create `src/zaxy/metacognition.py` with:

- `build_known_unknown_event(...)`
- `build_confidence_assessment_event(...)`
- `build_conflict_cluster_event(...)`
- `build_reverify_request_event(...)`
- `summarize_metacognition_events(events: Iterable[dict[str, Any]]) -> dict[str, Any]`
- strict validation for non-empty text, 0..1 confidence, positive source event seqs, 64-lowercase-hex source hashes, Eventloom citations with exact 12 or 64 hash characters, and allowed priorities `low|normal|high|urgent`.

Every builder must set `authority_status="non_authoritative"`. Known unknowns and reverify requests must set `status="open"`. Conflict clusters must set `resolution_status="unresolved"`.

- [ ] Step 4: Run tests

Run:

    pytest tests/test_metacognition.py --no-cov -q

Expected: all metacognition contract tests pass.

---

### Task 2: Add MemoryFabric Metacognition Services

Files:

- Modify: `src/zaxy/core.py`
- Modify: `tests/test_metacognition.py`
- Modify: `tests/test_reasoning_primitives.py`

- [ ] Step 1: Add failing MemoryFabric tests

Add async tests proving:

- `record_known_unknown(...)` appends `metacognition.unknown.recorded` and records `reasoning.primitive.called`.
- `get_claim_confidence(..., record_assessment=True)` appends `metacognition.confidence.assessed` after scoring cited support/conflict evidence.
- `list_confidence_trajectory(claim=...)` replays confidence assessments for a claim without requiring Neo4j.
- `list_reverification_needs(...)` returns open unknowns, low-confidence assessments, conflict clusters, and reverify requests.
- Confidence scoring excludes metacognition events, reasoning primitive observations, and belief proposals from support/conflict evidence.

Use embedded/Eventloom-only tests. Monkeypatch projection where necessary.

- [ ] Step 2: Run tests to verify failure

Run:

    pytest tests/test_metacognition.py tests/test_reasoning_primitives.py -k "metacognition or confidence" --no-cov -q

Expected: fail because `MemoryFabric` methods and assessment recording are missing.

- [ ] Step 3: Implement MemoryFabric services

Add methods:

    async def record_known_unknown(
        self,
        question: str,
        *,
        reason: str,
        source_events: list[dict[str, Any]],
        claim_key: str,
        gap_type: str = "missing_evidence",
        reverify_query: str | None = None,
        phase: str = "review",
        session_id: str = "default",
        actor: str = "zaxy-reasoning",
    ) -> dict[str, Any]: ...

    async def list_known_unknowns(
        self,
        *,
        session_id: str = "default",
        status: str = "open",
        limit: int = 10,
    ) -> dict[str, Any]: ...

    async def list_conflict_clusters(
        self,
        *,
        session_id: str = "default",
        unresolved_only: bool = True,
        limit: int = 10,
    ) -> dict[str, Any]: ...

    async def list_confidence_trajectory(
        self,
        claim: str,
        *,
        session_id: str = "default",
        limit: int = 10,
    ) -> dict[str, Any]: ...

    async def list_reverification_needs(
        self,
        query: str | None = None,
        *,
        session_id: str = "default",
        limit: int = 10,
        min_confidence: float = 0.7,
    ) -> dict[str, Any]: ...

Update `get_claim_confidence(...)` to accept `record_assessment: bool = True` and append `metacognition.confidence.assessed` when true. If support and conflict are both present, append `metacognition.conflict.clustered`. If confidence is below `min_confidence` or conflict count is non-zero, append `metacognition.reverify.requested`.

Implementation requirements:

- Replay Eventloom logs for list methods; do not require graph availability.
- Do not close or resolve unknowns automatically.
- Do not count metacognition events, `reasoning.primitive.called`, or `belief.update.proposed` as support evidence.
- Every method that is model-callable records `reasoning.primitive.called`.

- [ ] Step 4: Run tests

Run:

    pytest tests/test_metacognition.py tests/test_reasoning_primitives.py -k "metacognition or confidence" --no-cov -q

Expected: all Task 2 tests pass.

---

### Task 3: Add Typed Metacognition Projection and Checkout Diagnostics

Files:

- Modify: `src/zaxy/extract.py`
- Modify: `src/zaxy/checkout.py`
- Modify: `tests/test_extract.py`
- Modify: `tests/test_causal_checkout.py`
- Modify: `tests/test_checkout.py`

- [ ] Step 1: Add failing extractor and checkout tests

Tests must prove:

- `metacognition.unknown.recorded`, `metacognition.confidence.assessed`, `metacognition.conflict.clustered`, and `metacognition.reverify.requested` project to typed entities.
- Projected entities preserve `authority_status="non_authoritative"`, status/resolution fields, confidence, source event refs, and claim keys.
- Extractors reject `authority_status="authoritative"`.
- Checkout diagnostics include `metacognition.unknown_count`, `open_unknown_count`, `conflict_cluster_count`, `low_confidence_count`, and `reverify_needed_count`.
- Prompt guidance tells the model unresolved/high-uncertainty metacognition is diagnostic, not fact authority.

- [ ] Step 2: Run tests to verify failure

Run:

    pytest tests/test_extract.py tests/test_causal_checkout.py tests/test_checkout.py -k "metacognition or unknown or reverify" --no-cov -q

Expected: fail because typed extraction and diagnostics are missing.

- [ ] Step 3: Implement typed extraction

Add extractors in `src/zaxy/extract.py`:

- `@register("metacognition.unknown.recorded")`
- `@register("metacognition.confidence.assessed")`
- `@register("metacognition.conflict.clustered")`
- `@register("metacognition.reverify.requested")`

Use entity types:

- `known_unknown`
- `confidence_assessment`
- `conflict_cluster`
- `reverify_request`

Use deterministic entity names from event payload IDs. Use the existing source-event snapshot helpers for backend-safe `source_event_refs`, `source_event_seqs`, and `source_event_hashes`.

- [ ] Step 4: Implement checkout diagnostics/guidance

Add deterministic diagnostics in `src/zaxy/checkout.py`:

    diagnostics["metacognition"] = {
        "unknown_count": ...,
        "open_unknown_count": ...,
        "conflict_cluster_count": ...,
        "unresolved_conflict_count": ...,
        "low_confidence_count": ...,
        "reverify_needed_count": ...,
        "authority_status": "non_authoritative",
    }

Guidance must include:

- unresolved known unknowns require re-verification or user clarification;
- confidence assessments are trajectory evidence, not truth;
- conflict clusters are diagnostic until resolved by a separate authority path.

- [ ] Step 5: Run tests

Run:

    pytest tests/test_extract.py tests/test_causal_checkout.py tests/test_checkout.py -k "metacognition or unknown or reverify" --no-cov -q

Expected: pass.

---

### Task 4: Harden Procedural Planning Lane

Files:

- Modify: `src/zaxy/core.py`
- Modify: `src/zaxy/checkout.py`
- Modify: `src/zaxy/mcp_server.py`
- Modify: `tests/test_reasoning_primitives.py`
- Modify: `tests/test_checkout.py`
- Modify: `tests/test_mcp.py`

- [ ] Step 1: Add failing procedural tests

Tests must prove:

- `retrieve_similar_procedures(...)` returns separate `applicable`, `diagnostic`, and `excluded` buckets.
- Only validated/revised/accepted cited procedures are applicable.
- Pending, deferred, deprecated, contradicted, stale, superseded, rejected, conflicted, uncited, and failed-outcome procedures are excluded or diagnostic, not operational instructions.
- Checkout diagnostics include `procedural_memory.applicable_count`, `diagnostic_count`, `excluded_count`, `rollback_candidate_count`, `contradiction_count`, and `excluded_reasons`.
- `memory_skill` accepts and persists `failure_modes`, `rollback`, and `contradiction_reason`.

- [ ] Step 2: Run tests to verify failure

Run:

    pytest tests/test_reasoning_primitives.py tests/test_checkout.py tests/test_mcp.py -k "procedure or skill" --no-cov -q

Expected: fail because the procedural planning lane is still flat and `memory_skill` does not pass all rollback fields.

- [ ] Step 3: Implement procedural buckets

Update helper logic near `_procedure_contexts`:

- return `{"applicable": [...], "diagnostic": [...], "excluded": [...], "excluded_reasons": {...}}`;
- require citation for applicable procedures;
- applicable statuses: `validated`, `revised`, `accepted`;
- diagnostic statuses: `proposed`, `pending`, `deferred`;
- excluded statuses: `rejected`, `conflicted`, `deprecated`, `contradicted`, `stale`;
- exclude `valid_to` closed and `superseded_by` contexts;
- preserve `rollback`, `failure_modes`, `contradiction_reason`, and latest citation.

Update `retrieve_similar_procedures` to keep `procedures` as a backward-compatible alias for `applicable`, and add `applicable`, `diagnostic`, `excluded`, and `procedural_memory`.

- [ ] Step 4: Extend Skill Memory MCP helper

In `src/zaxy/mcp_server.py`, extend `memory_skill` schema and handler to accept:

- `failure_modes: list[str]`
- `rollback: str`
- `contradiction_reason: str`

Pass these fields into the payload for `skill.contradicted`, `skill.deprecated`, and other actions when supplied. Preserve existing validation for list fields.

- [ ] Step 5: Add checkout procedural diagnostics

Add/extend checkout diagnostics:

    diagnostics["procedural_memory"] = {
        "applicable_count": ...,
        "diagnostic_count": ...,
        "excluded_count": ...,
        "rollback_candidate_count": ...,
        "contradiction_count": ...,
        "excluded_reasons": {...},
        "authority_status": "non_authoritative",
    }

Prompt guidance must say applicable procedures are planning guidance, not authoritative facts, and rollback/contradiction candidates should be avoided or explicitly reviewed.

- [ ] Step 6: Run tests

Run:

    pytest tests/test_reasoning_primitives.py tests/test_checkout.py tests/test_mcp.py -k "procedure or skill" --no-cov -q

Expected: pass.

---

### Task 5: Add CLI, MCP, Docs, and Beta.2 Guardrail

Files:

- Modify: `src/zaxy/__main__.py`
- Modify: `src/zaxy/mcp_server.py`
- Modify: `src/zaxy/reasoning_benchmark.py`
- Modify: `tests/test_cli.py`
- Modify: `tests/test_mcp.py`
- Modify: `tests/test_reasoning_benchmark.py`
- Modify: `docs/examples/mcp-tool-contract.json`
- Modify: `docs/mcp.md`
- Modify: `docs/graph-schema.md`
- Modify: `docs/benchmarks.md`
- Modify: `docs/agent-events.md`
- Modify generated `site/docs/*.html`

- [ ] Step 1: Add failing public-surface tests

Add CLI tests for:

- `zaxy memory reasoning record-unknown`
- `zaxy memory reasoning known-unknowns`
- `zaxy memory reasoning confidence-trajectory`
- `zaxy memory reasoning reverify-needed`
- `zaxy memory reasoning plan-from-procedures`

Add MCP schema/handler/dispatch tests for:

- `memory_record_known_unknown`
- `memory_known_unknowns`
- `memory_confidence_trajectory`
- `memory_reverification_needs`
- `memory_plan_from_procedures`

Add guardrail tests proving beta.2 scoring includes:

- metacognition observability;
- reverify open status;
- procedural citation presence;
- planning phase match;
- authority boundary.

- [ ] Step 2: Run tests to verify failure

Run:

    pytest tests/test_cli.py tests/test_mcp.py tests/test_reasoning_benchmark.py -k "metacognition or unknown or reverify or trajectory or plan_from_procedures or beta2" --no-cov -q

Expected: fail because public surfaces and guardrail are missing.

- [ ] Step 3: Implement CLI and MCP surfaces

CLI command names:

- `record-unknown`
- `known-unknowns`
- `confidence-trajectory`
- `reverify-needed`
- `plan-from-procedures`

MCP tool names:

- `memory_record_known_unknown`
- `memory_known_unknowns`
- `memory_confidence_trajectory`
- `memory_reverification_needs`
- `memory_plan_from_procedures`

All CLI/MCP handlers must instantiate configured `MemoryFabric`, call the matching core method, close the fabric in `finally`, and return JSON-compatible dicts.

- [ ] Step 4: Implement beta.2 guardrail

Add `score_metacognition_guardrail(...)` or extend `score_reasoning_guardrail(...)` with explicit beta.2 fields:

- `observable_metacognition`
- `open_reverify_status`
- `procedural_citation_presence`
- `planning_phase_match`
- `authority_boundary`
- `score`

The scorer must inspect contract fields only. It must not score task answers or benchmark labels.

- [ ] Step 5: Update docs and generated site

Document:

- metacognitive events are diagnostic and non-authoritative;
- confidence trajectories are append-only and do not overwrite facts;
- reverify requests remain open until separately resolved;
- procedural planning lane separates applicable/diagnostic/excluded procedures;
- rollback/contradiction diagnostics are avoid/review signals;
- beta.2 guardrail is internal/project-defined, not external validation.

Run:

    python scripts/build-site-docs.py
    scripts/validate-docs.sh --root .

- [ ] Step 6: Run public-surface tests

Run:

    pytest tests/test_cli.py tests/test_mcp.py tests/test_reasoning_benchmark.py -k "metacognition or unknown or reverify or trajectory or plan_from_procedures or beta2" --no-cov -q

Expected: pass.

---

## Final Regression Gate

After all tasks:

    pytest \
      tests/test_metacognition.py \
      tests/test_reasoning_primitives.py \
      tests/test_reasoning_benchmark.py \
      tests/test_causal_checkout.py \
      tests/test_checkout.py \
      tests/test_extract.py \
      tests/test_cli.py \
      tests/test_mcp.py \
      -k "metacognition or unknown or reverify or trajectory or procedure or skill or beta2" \
      --no-cov -q

    pytest tests/test_checkout.py tests/test_graph.py tests/test_mcp.py tests/test_extract.py --no-cov -q

    ruff check \
      src/zaxy/metacognition.py \
      src/zaxy/reasoning_primitives.py \
      src/zaxy/reasoning_benchmark.py \
      src/zaxy/core.py \
      src/zaxy/__main__.py \
      src/zaxy/mcp_server.py \
      src/zaxy/checkout.py \
      src/zaxy/extract.py \
      tests/test_metacognition.py \
      tests/test_reasoning_primitives.py \
      tests/test_reasoning_benchmark.py \
      tests/test_causal_checkout.py \
      tests/test_checkout.py \
      tests/test_cli.py \
      tests/test_mcp.py \
      tests/test_extract.py

    scripts/validate-docs.sh --root .

    python -m zaxy benchmark-compare \
      reports/benchmarks/longmemeval-500-publish-20260607/live-benchmark.json \
      --backend zaxy-checkout \
      --min-mean-score 0.95 \
      --min-answer-recall-at-5 0.90 \
      --min-recall-at-5 0.99 \
      --min-citation-coverage 1.0 \
      --max-p95-ms 2500 \
      --max-p99-ms 3000

Expected:

- focused beta.2 tests pass;
- core checkout/graph/MCP/extract regressions pass;
- ruff clean;
- docs validation passes;
- cached LongMemBench guardrail passes without implementation code tailored to that artifact.

## Self-Review Notes

Spec coverage:

- Known unknowns: Tasks 1-3 and Task 5.
- Conflicting evidence clusters: Tasks 1-3 and Task 5.
- Confidence trajectories: Tasks 1-3 and Task 5.
- Re-verification surfaces: Tasks 1-3 and Task 5.
- Procedural memory first-class planning lane: Task 4 and Task 5.
- Rollback/contradiction diagnostics: Task 4 and docs in Task 5.
- No authority promotion: every task includes explicit non-authoritative checks.
- No benchmark tailoring: final benchmark comparison is a guardrail only.

Known risks:

- Initial conflict clustering is deterministic and conservative; it is a diagnostic primitive, not a belief-revision engine.
- Replay-derived metacognition lists may need future pagination for very long sessions.
- Procedural planning buckets depend on existing Skill Memory and consolidation projection quality.