# Heuristic System Framework

## Purpose

agent-kit is an agent-facing Heuristic System framework. It gives agents a
maintainable operating layer made of skills, scripts, runbooks, evidence
records, tests, and guardrails. The framework does not train model weights; it
lets agents improve the working system around the model by turning workflow
results into clearer skill policy, validation checks, scripts, tests,
primitives, or runbook updates.

## Read This When

Read this document before creating, updating, reviewing, or removing tracked
skills; changing skill contracts, scripts, references, tests, or workflow
primitives; designing new evidence, failure-handling, or recovery conventions;
or explicitly using a skill whose behavior depends on retained evidence. It is
required for `skill-dev` context through `agent-docs` and recommended for
explicit skill use.

## System Shape

| Part | Role |
| --- | --- |
| Skills | Human-readable workflow policy, judgment boundaries, and usage contracts. |
| Scripts and primitives | Deterministic execution, validation, evidence capture, and guardrails. |
| Tests and checks | Regression protection for old capabilities and workflow contracts. |
| Evidence records | Redacted records of failures, waivers, validation, review, and browser/API activity. |
| Runbooks | Stable operating knowledge that should outlive one session. |
| Memory | Personal setup and recurring preferences only; not project state or factual proof. |

## Core Loop

When a skill workflow produces new operational knowledge: run the skill within
project rules, capture the relevant result or failure, diagnose from concrete
evidence before changing policy or code, fix or work around within scope,
promote repeated lessons into a durable location, and compress accumulated
local patches into simpler contracts, tests, scripts, or runbooks. Goal: not
record everything — preserve useful learning a future agent can verify and
reuse.

## Activation And Triage

Heuristic triage activates from one signal: a workflow result that failed or
felt wrong. Whether a named skill was active only changes which deterministic
record applies — it does not gate triage.

- `skill-usage.record.v1` is the strict envelope for named-skill workflows:
  create it only when a named skill is invoked or selected and the workflow
  performs edits, tool/API calls, validation, delivery, external lookup, or
  durable artifact creation. Outside a named skill, no usage record is owed
  and triage proceeds without one.
- `exit_code != 0`, stderr output, a single retry, or a corrected typo starts
  judgment, not persistence. Classify the case before writing durable
  artifacts.

| Outcome | Use when |
| --- | --- |
| Ignore | The issue was a transient typo, wrong cwd, obvious authoring miss, or immediate fix with no reusable lesson. |
| Summarize | The friction helps explain the current result but does not justify a retained repo artifact. |
| Retain or promote | The lesson is important, unresolved, repeated, skill-contract relevant, or reusable by future agents. |

Use this checklist before retaining observed friction. This is the canonical
triage checklist — other docs link here instead of restating it.

- Did the same command, validation, or workaround require repeated retries?
- Did documented behavior and actual behavior disagree?
- Was the error output unclear for the action the agent needed to take?
- Did the fix require a semantic workaround rather than a simple formatting
  correction?
- Did the friction happen inside a named skill workflow or contradict a skill
  contract?
- Would a future agent benefit from a test, runbook, issue, inbox entry,
  primitive change, or skill policy update?

Skill-contract relevance lowers the retention threshold. If friction happens
inside an active named skill workflow that already requires a
`skill-usage.record.v1` envelope, record the command/script/dependency failure in
that envelope's `failures` list and then decide whether a curated follow-up is
also warranted. Do not commit raw stderr or turn ordinary authoring
mistakes into retained artifacts.

## Promotion Ladder

Promote workflow lessons by durability:

| Signal | Preferred durable form |
| --- | --- |
| One-off execution result | Runtime evidence or final response summary. |
| Important unresolved workflow gap | Curated `heuristic-system/error-inbox/` entry. |
| Repeated or cross-skill failure | Focused test, script fix, shared runbook, primitive, or `heuristic-system/operation-records/` entry. |
| Stable project policy | `AGENTS.md`, project docs, or repo-local runbook. |
| Personal recurring preference | Memory, when allowed by the memory policy. |

Do not promote secrets, raw credentials, unredacted logs, or temporary task
state into durable docs or memory.

## Error Inbox

Use curated `heuristic-system/error-inbox/` entries when an important workflow
gap is observed but not fixed in the same turn. Keep raw runtime records in
their evidence location; commit only a short tracker entry with signal,
evidence pointer, impact, workaround, promotion criteria, and next action.

The active lifecycle is `open | promoted | wontfix`. Progress between
`open` and `promoted` is represented by `Next Action` text and linked plan,
issue, or PR references — not by extra enum values. Older entries may carry
legacy lifecycle values; the primitive reads them but does not accept them on
new writes.

After a gap is fixed, validated, and has no remaining next action, keep its
status as `promoted` or `wontfix` and move the entry under
`heuristic-system/error-inbox/archive/YYYY/` so the active inbox does not
become a stale backlog. Archiving does not delete curated evidence — it moves
a completion record out of the active queue.

Use the `heuristic-error-inbox` workflow skill to create, verify, deduplicate,
triage, update, or archive these entries.

## Case Layout

Inbox cases and operation records are stored as per-case folders so the
curated tracker and any redacted evidence can live together in vcs:

- Inbox case: `heuristic-system/error-inbox/<slug>/ENTRY.md` plus
  `<slug>/evidence/<artifact>.md`.
- Archived inbox case:
  `heuristic-system/error-inbox/archive/YYYY/<slug>/ENTRY.md` plus
  `<slug>/evidence/` (when present).
- Operation record:
  `heuristic-system/operation-records/<slug>/RECORD.md` plus
  `<slug>/evidence/`.

Plans stay in `docs/plans/<slug>/` under their own lifecycle. A case folder
may reference a plan from `ENTRY.md` or `RECORD.md`, but it does not
duplicate plan content. Use `heuristic-inbox ingest-evidence` to add
a redacted artifact to a case; the primitive rejects raw `skill-usage.record.json`
files, content matching token-like patterns, files above the configured size
limit (default 64 KiB), and absolute `$HOME` paths that have not been
rewritten to `<workspace>`. Runtime evidence is still not automatically a
repo artifact — only redacted, primitive-accepted excerpts live inside a case
folder.

The ingest redaction guardrail applies to files under `<slug>/evidence/`
written through `ingest-evidence`. Starting with `heuristic-inbox 0.10.2`,
`verify` also scans `ENTRY.md` and `RECORD.md` body text for absolute `$HOME`
paths. Non-strict verify reports body warnings while preserving historical
compatibility; `verify --strict` escalates body findings into violations and
`ok=false`. New or updated retained cases should pass `heuristic-inbox verify
--strict` before they are committed or reported complete.

## Operation Records

Use `heuristic-system/operation-records/` for retained workflow failures
important enough to prove the heuristic loop operated across a broader
workflow surface. Keep raw runtime records in their evidence location; commit
only the compressed record that names signal, evidence, diagnosis, promotion
decision, durable fix, validation, and retention outcome.

Operation records are not required for every promoted inbox entry. A focused
test, script fix, runbook update, or skill policy update is enough when it
captures the local lesson. Prefer an operation record when the signal is
repeated, cross-skill, audit-worthy, or useful as proof that retained evidence
was compressed into durable system behavior.

## Compression Rule

Heuristic Systems decay when they only grow. When a skill accumulates several
local exceptions, retries, or failure notes: group by root cause, keep the
smallest stable rule that explains the group, replace repeated prose with a
test, guardrail, or script when practical, and archive resolved
`heuristic-system/error-inbox/` entries once they are `promoted` or `wontfix`
with no remaining next action. Add a dedicated `heuristic-compression-review`
skill (or other broader heuristic-* skills) only after several related
archived inbox or operation records prove a repeatable command surface; until
then, keep compression work inside the narrow workflow skill or implementation
plan that owns the records.

## Boundaries

- Skills own workflow framing, judgment, and repo-local policy; primitives own
  deterministic record writing, validation, redaction, and machine-checkable
  execution.
- `agent-docs` owns read-first context selection and hard-gate preflight.
- Runtime evidence is not automatically a repo artifact — commit only curated
  evidence or docs that project policy expects to retain.

## Relationship To Skill Usage Recording

The implemented convention for automatic skill usage records lives at
`docs/runbooks/skills/SKILL_USAGE_RECORDING_V1.md`. Use the nils-cli
`skill-usage` primitive for deterministic record writing, validation,
redaction, and linked evidence. This document defines the durable framework
concept and the required read-first boundary for skill development.
