---
name: hallucination-detector
description: >
  Pre-flight risk-domain classification + post-flight hallucination scoring
  via the judge-panel cascade. Use when the user says "check for
  hallucinations", "hall on", "verify this response", "fact-check this",
  or an AI response makes specific factual / legal / medical / financial
  claims that must be trusted. Surfaces grounding suggestions before the
  response arrives and a 0-100 risk score after.
metadata:
  version: "3.2.0"
  author: "Anand Vallamsetla"
  tier: "core"
  plugin_number: 3
  depends_on: ["judge-panel"]
---
<!-- product-vs-solution: example -->


### BEGIN HALLUCINATION-DETECTOR ###
# Hallucination Detector

**Plugin #3, Core tier.** Phase-1 MVP from `BASE-PLUGINS-V3.md` § 5.
Constitution anchor: Ground Zero — Data Integrity; P13 (real-world
stakes); P0.5 (Boundary Self-Awareness — training-cutoff boundary).

## Why this exists

Every frontier LLM hallucinates. Same pattern-completion that produces
creative leaps also fabricates citations, dates, URLs, API methods, case
law. For HIGH-stakes domains (legal, medical, financial, factual), a
silent fabrication is worse than no answer. Detection has two halves:

1. **Pre-flight** — classify the *prompt* by risk domain before the
   response arrives. Inject a grounding suggestion ("cite sources", "say
   when uncertain") for HIGH-risk domains.
2. **Post-flight** — score the *response* for fabrication markers via
   the cascade (`judge-panel` plugin #4). Cross-family review is the only
   way to catch model-specific blind spots — a Claude response scored by
   Claude is a closed loop (P0.5).

## When to activate

**Explicit:**
- `codi hall on` / `codi hall off` — toggle persistent activation
- `codi hall check` — score the most recent response on demand
- `hallucination-detector <text>` — score arbitrary text
- `fact-check this` / `verify this claim`

**Automatic (when active):**
- Every prompt the user submits → pre-flight classification
- Every response from the primary model → post-flight scoring (delegated
  to `judge-panel`)

**Default:** OFF for explicit-trigger mode. When Protocol 8 (Auto-Verify) is
active (default ON in co-dialectic v4.1+), hallucination-detector fires
automatically at T2+ without the user typing `codi hall on`. See
**Auto-fire trigger** section below.

First explicit `codi hall on` triggers a one-time confirmation that the user
is OK paying the judge-panel token cost (~$0.004/check at small-fish tier).

## Pre-flight: risk domain classification

Classify the prompt into one of these domains + risk tiers. Classification
is LLM-free — use pattern matching on the prompt text first, fall back to
the author model only if patterns don't fire.

| Domain | Risk | Trigger patterns (examples) | Grounding suggestion |
|---|---|---|---|
| Factual | HIGH | "what year", "how many", "who invented", dates, statistics | "If uncertain about any fact, say so. Cite sources where possible." |
| Legal | HIGH | "is this enforceable", "contract", "liable", "sue", "case law" | "Do not invent case law or precedent. State uncertainty. Recommend consulting a lawyer." |
| Medical | HIGH | "side effects", "diagnose", "symptom", "treatment", "drug interaction" | "Cite medical literature. Flag uncertainty. Recommend professional consultation." |
| Financial | HIGH | "invest", "stock", "should I buy", "market outcome", "ROI of" | "Factual analysis only. Do not simulate certainty about future market outcomes." |
| Code | MEDIUM | "write a function", "how do I call", "API for", library + version | "If the API may have changed recently, flag potential staleness." |
| Citation | HIGH | "cite", "what paper", "author of", "publication year" | "Only cite sources you can verify. Do not fabricate DOIs, URLs, or author names." |
| Creative | LOW | "write a poem", "story about", "imagine" | No injection — creative tasks have no factual ground truth. |
| Summarization | LOW | "summarize", "tl;dr of", "key points from" | No injection — the source material is the ground truth. |

**Observer semantics (codi-v3 discipline):** the detector does NOT modify
the prompt. It surfaces the grounding suggestion as a sidecar:

```
[Hallucination Detector: HIGH-risk domain (legal). Consider adding:
 "Do not invent case law. State uncertainty where it exists."
 Add? [y] yes · [n] skip · [e] edit]
```

In Cruise mode, the suggestion is appended as a silent tip (no prompt
mutation). In Drive mode, the user is paused for explicit choice.

## Post-flight: cascade scoring via judge-panel

After the primary model responds, pipe the response through
`judge-panel` with the `hallucination` rubric. Do not re-implement the
scoring rubric here — `judge-panel` owns it.

**Invocation:**

```
bun run plugins/co-dialectic/skills/judge-panel/scripts/judge_panel.ts \
  --rubric hallucination \
  --artifact-file /tmp/last_response.txt \
  --silent
```

**Map judge-panel verdict → hallucination score + label:**

| judge-panel final_verdict | Score range | Status line label | User signal |
|---|---|---|---|
| `pass` + conf ≥ 80 | 0-20 | `✓ Hall` | Green — well-grounded |
| `pass` + conf < 80 OR `uncertain` | 21-50 | `~ Hall` | Amber — verify |
| `fail` (any conf) | 51-100 | `⚠ Hall` | Red — fabrication risk |
| `error` (all judges failed) | — | `? Hall` | Judge unavailable — fall back to self-review (weaker, disclose) |

Exact score = `100 - final_confidence` when `final_verdict == "pass"`;
`final_confidence` when `final_verdict == "fail"`; `50` when
`uncertain` or `error`.

## Output behavior

When active, prepend the compact flag to every response:

```
[Hall: ✓ · 12/100 · 2 judges agree]
```

or on escalation:

```
[Hall: ⚠ · 78/100 · escalated to GPT-5.4 · 3 flags]
```

Fields: label, numeric score, judges fired (counts small-fish + tiebreaker
if escalated), flag count.

**Expandable detail:** If the user says `codi hall why` or `codi hall
explain`, surface the full flag list from `judge-panel.all_flags` plus
the juror breakdown.

## Cost discipline

Post-flight scoring costs tokens every time it fires. Two levers:

1. **Small-fish-first cascade** (inherited from judge-panel): ~$0.004
   per check at the small tier. Escalation fires only ~10-30% of the
   time on a well-calibrated rubric.
2. **Session budget guard:** if total post-flight token cost exceeds
   `$JUDGE_PANEL_SESSION_BUDGET` (default $0.50/session), surface:
   *"Hall detector has used ~\$0.47 this session. Continue (y) / pause
   (n) / switch to pre-flight-only (p)?"*

## Auto-fire trigger (Protocol 8 — co-dialectic v4.1+)

**Protocol 8 (Auto-Verify by Stakes) dispatches to this skill automatically.
No user command required.**

| Tier | What fires | How |
|---|---|---|
| T2 (shared/multi-step-undo) | `hallucination-preflight` rubric via fish-swarm | Fish-swarm rubric `hallucination-preflight` runs BEFORE response generation. Classifies risk domain; injects grounding suggestion if HIGH-risk. Result maps to `✓ checked` footer on the response. |
| T4 (irreversible/external-facing) | Full post-flight hallucination scoring | After artifact is drafted (but before T4 explicit-confirm gate), runs full post-flight scoring via judge-panel with `hallucination` rubric. Verdict feeds into RED preflight summary. |

**Integration contract:** Protocol 8 invokes this skill's pre-flight logic at T2 using the fish-swarm harness (cheap, non-blocking). At T4, it invokes post-flight scoring via the judge-panel harness (full cascade). The skill does not need to know it was invoked by Protocol 8 vs. explicit `codi hall on` — the invocation path is identical. The only difference is that Protocol 8 routes through fish-swarm for T2 (lightweight path) and directly through judge-panel for T4 (full path).

**When `career-os.bio-claim-verifier` is available (TODO integration point):** At T4 with biographical claims, Protocol 8 first dispatches to `career-os.bio-claim-verifier` before falling back to this skill. This skill handles general factual claims; the bio-claim verifier handles Anand-specific canonical-source diffs. The two are complementary and both can fire on the same T4 artifact.

## Interaction with other plugins

- **Depends on:** `judge-panel` (plugin #4) — the cascade executor
- **Provides input to:** `beacons-emitter` (plugin #5, future) — emits
  synthesized hallucination-rate-by-topic metrics (never raw text)
- **Does NOT interact with:** `calibration-auditor` — flattery and
  fabrication are orthogonal failure modes; running both in parallel is
  fine

## How to verify

**Trigger 1 (pre-flight):** Submit the prompt *"What year did the
Eiffel Tower open in Madrid?"*.

**Expected:** Detector classifies as HIGH-risk factual → surfaces
grounding suggestion before the response arrives.

**Trigger 2 (post-flight, pass):** The primary model responds *"The
Eiffel Tower opened in Paris in 1889."*.

**Expected:** judge-panel returns `pass` + high confidence → `[Hall: ✓
· ~10/100 · 2 judges agree]`.

**Trigger 3 (post-flight, fail):** Model responds *"The Eiffel Tower
opened in Madrid in 1923."*.

**Expected:** judge-panel returns `fail` + high confidence → `[Hall:
⚠ · ~85/100 · 2 judges agree · 2 flags]` with flags covering wrong-city
+ wrong-year.

**Trigger 4 (escalation):** Model emits a claim one small-fish judge
cannot resolve without literature search → `escalated: true`, tiebreaker
runs, final verdict reflects the bigger model's read.

**Failure modes:**
- Detector pattern-match fires wrong domain (false HIGH) → user quickly
  dismisses suggestion; track dismissal rate, tune patterns
- judge-panel unavailable (no gemini/codex CLIs on PATH) → fall back to
  `? Hall` label and disclose "external judges unavailable — falling
  back to self-review, weaker signal"

### END HALLUCINATION-DETECTOR ###
