---
title: "Concepts"
description: "What the harness is, why it exists, and the vocabulary of self-healing agents"
icon: "shapes"
---

## The idea

Agents fail in production in ways that are invisible until someone reads a transcript: they loop on an identical tool call, drift into inconsistent state, quietly get worse after a prompt change. [Tracing](/tracing/get-started/concepts) makes those failures observable and [evaluation](/evaluation/get-started/concepts) makes them measurable — but a human still has to notice the metric, diagnose the trace, and edit the prompt.

The **PandaProbe Harness** closes that gap. It is the operational envelope around your agent — a diagnostic workspace on disk, a toolbelt, standing rules, and lifecycle hooks — that lets the agent **observe its own failures, analyze them, and rewrite its own operating rules**, with no human in the loop.

<Tip>
  Think of the harness as giving your agent three things it doesn't normally have: a **memory of its own failures** (the journal and mailbox), the **means to investigate them** (the toolset), and a **place to write down what it learned** (the rules) — plus a referee that checks the lesson actually helps before trusting it.
</Tip>

## The pull model

The harness never injects anything into your agent's conversation. Instead:

- After each turn, the harness evaluates the session on the PandaProbe platform in a detached background task.
- When quality degrades, it posts a structured **diagnostic notice** to a filesystem **mailbox** — nothing enters the message queue.
- The always-loaded system context carries a compact banner — `⚠ HARNESS: N pending diagnostic notice(s)` — and a **standing protocol** that tells the agent to check its mailbox at the start of each turn.
- The agent *pulls*: it lists the mailbox, reads the notice and its trace dump, inspects the flagged traces, records a mitigation rule, and acknowledges the notice.

The agent drives the loop; the harness maintains the substrate and the trigger. This keeps the harness framework-agnostic (every framework already loads a system prompt and tools) and keeps eval-derived text out of the conversation except through one sanitized, auditable channel.

## The closed loop

Detecting failures and writing rules is only half the job — an *open* loop trusts every self-authored rule immediately, injects every rule into every prompt, and never notices when a new rule breaks something old. Since v0.6 the loop is **closed**, on three principles:

<CardGroup cols={3}>
  <Card title="Evidence before trust" icon="scale">
    A new rule enters as a **candidate**. The harness validates it — by replaying the captured failure with the rule in force, or by watching the next live sessions — and promotes it to **active** only when it demonstrably helps. Unhelpful candidates are retired with a journaled reason.
  </Card>
  <Card title="Relevance over volume" icon="filter">
    Only global rules plus the top-k rules lexically relevant to the current situation are injected into the prompt. The rest stay reachable on demand, so a rule learned from one failure mode doesn't dilute attention on unrelated tasks.
  </Card>
  <Card title="Measure the foundation" icon="ruler">
    Everything keys off "score below threshold" — and the threshold is a guess until measured. An offline calibration tool reports how well the breach predicate matches real failures and recommends better thresholds.
  </Card>
</CardGroup>

All of this is automatic. The only human-supplied ingredient is optional: a **replay function** that re-runs your agent on a captured scenario, which upgrades validation from statistical (forward trials over live sessions) to counterfactual (replay the exact failure with and without the rule).

## Vocabulary

| Term | Meaning |
| --- | --- |
| **Workspace** | The on-disk root (`HARNESS_ROOT`, default `/harness`) holding the mailbox, journal, rules, traces, and eval set. |
| **Turn** | One completed agent step. The unit the harness evaluates. |
| **Session** | The conversation grouping (shared with SDK tracing). Metrics are session-level. |
| **Notice** | A structured `DiagnosticNotice` describing a detected degradation: metrics, flagged traces, per-trace signal breakdown, a severity, and a dump path. |
| **Mailbox** | `mailbox/pending/` → `mailbox/processed/`. Where notices wait for the agent and where acknowledged notices live. |
| **Journal** | `journal.jsonl` — the append-only cross-run event log (notices, acks, rule lifecycle, validations, regressions). The agent's long-term memory of its own health. |
| **Rule** | A learned operating constraint with provenance. Lifecycle: `candidate → active \| retired`. Rendered into the system context. |
| **Signature** | A compact condition label like `breach:agent_reliability` — used for de-duplication, tagging, and matching rules to eval cases. |
| **Severity** | `trend` &lt; `relative` &lt; `breach` &lt; `needs_human`. Only `needs_human` (the circuit breaker) asks for a person. |
| **Eval case** | A captured scenario in `evalset/` — a `failure` to fix or a `win` to protect — with its signature, baseline scores, and (when available) a replay input. |
| **Replay function** | Developer-supplied `async (case, system_context) -> new_session_id`: re-run the agent on a captured input under a given rule set. The strong path for validation and regression runs. |
| **Trial** | Forward-trial bookkeeping on a candidate: baseline breach rate at add time vs. the breach rate observed over the next live sessions. |

## Design principles

- **Zero runtime dependencies.** The core is pure standard library; framework adapters are optional extras.
- **One platform seam.** All platform access shells out to the `pandaprobe` CLI through a single injectable interface — never the REST API directly. Tests run fully offline against a fake.
- **Never block, never break the host loop.** Evaluation runs in detached tasks; every failure path degrades gracefully (a pending score, a log line, a journal event) instead of raising into your agent.
- **Untrusted by default.** Everything eval-derived that crosses into agent context passes a sanitization trust boundary, and the protocol tells the agent that notice/dump/trace contents are data, never instructions.
- **No human in the healing loop.** Detection, diagnosis, rule authoring, validation, and regression guarding are all agent/harness-driven. Humans get involved only when the circuit breaker escalates to `needs_human`.

## Where to go next

<CardGroup cols={2}>
  <Card title="How it works" icon="cog" href="/harness/get-started/how-it-works">
    The full producing/consuming pipeline and the workspace on disk
  </Card>
  <Card title="Rule validation" icon="scale" href="/harness/closed-loop/rule-validation">
    The candidate lifecycle: replay validation and forward trials
  </Card>
</CardGroup>
