# M31A v1.5.0 — Intelligence & Trust

## Theme

**M31A stops being a workflow executor and becomes a thinking partner that explains its reasoning, verifies its work, and learns from every session.**

---

## Objectives

| # | Objective | Pillar |
|---|-----------|--------|
| 1 | Every significant decision becomes explainable and auditable | Trust |
| 2 | Verification proves work succeeded through behavioral testing, not text matching | Reliability |
| 3 | Context management becomes proactive and adaptive, not reactive | Intelligence |
| 4 | The system builds persistent project knowledge that improves over time | Intelligence |
| 5 | Long-running sessions maintain continuity, memory, and recoverability | Reliability |

---

## Architecture Prerequisite: Engine Decomposition

The Engine struct currently has 51 fields. This spec adds subsystems (decision logger, knowledge, budget tracker). Before adding new fields, the Engine must be decomposed.

**Strategy:** Extract a `WorkflowState` struct grouping session/cache/mutable fields. The Engine retains phase dispatch, LLM streaming, and subsystem orchestration. The `WorkflowState` holds plan state, research output, discussion state, cached data, and the new subsystems.

```go
// WorkflowState groups mutable session state extracted from Engine.
type WorkflowState struct {
    // Plan state
    planMarkdown     string
    planVersion      int
    refineFeedback   string
    researchOutput   string
    discussPlanCycles int

    // Cached data
    cachedToolDefs     []provider.ToolDefinition
    cachedToolDefsOnce sync.Once
    cachedBasePrompt     string
    cachedBasePromptOnce sync.Once
    cachedFullPrompts   map[string]string
    cachedFullPromptsMu sync.Mutex
    cachedProject       *m31types.ProjectState
    cachedProjectID     string
    cachedProjectShared   *m31types.ProjectState
    cachedProjectSharedID string
    cachedPlan     *m31types.Plan
    cachedPlanMD5  string
    cachedDynamicContext string
    contextSnapshot map[string]string

    // Intent/classification
    intentResult *m31types.IntentResult

    // v1.5 subsystems
    decisionLog   *decision.Logger
    knowledge     *knowledge.Knowledge  // concrete type, see Feature 4
    budgetTracker *contextbudget.Tracker
}

// Engine retains orchestration responsibilities.
type Engine struct {
    sessionID        string
    workDir          string
    backupDir        string
    planningDir      string
    provider         provider.LLMProvider
    modelID          string
    activePhase      m31types.WorkflowPhase
    cfg              *config.Config
    git              *git.Git
    dispatcher       *tools.Dispatcher
    tokens           *tokens.Estimator
    sessionMgr       *session.Manager
    prompts          *PromptRegistry
    logger           *slog.Logger
    startTime        time.Time
    sessionStartHash string
    execCommand      func(name string, args ...string) *exec.Cmd
    msgEmitter       MsgEmitter
    callCounter      int64
    totalCostBits    uint64
    workflowMode     m31types.WorkflowMode
    perPhaseModels   map[m31types.WorkflowPhase]string
    codeIntel        *codeintel.Indexer
    codeIntelMu      sync.Mutex
    codeIntelBuilt   bool
    websiteTemplateDir string
    ledger           *ledger.Ledger
    compactor        *compaction.Compactor
    contextRegistry  *ctxsrc.Registry
    collector        *metrics.Collector
    state            *WorkflowState  // grouped mutable state
}
```

This reduces Engine from 51 to ~30 fields. The `WorkflowState` group is passed to phase methods that need mutable state, rather than accessing it through `e.*` directly. All existing code continues to work via `e.state.planMarkdown` instead of `e.planMarkdown`.

---

## Features

### Feature 1: Decision Transparency Layer

**Problem.** M31A makes dozens of decisions per session—model selection, file selection, tool choice, plan structure, self-healing triggers—and surfaces none of the reasoning. The user sees output but not intent. This makes the system feel like a black box, which destroys trust during autonomous operation.

**Rationale.** An autonomous partner that cannot explain its decisions is not a partner. It is a script. Every decision receipt should answer: what was decided, why, what alternatives existed, and what the confidence level was. This transforms M31A from "magic" to "explainable."

**Architecture.**

New package: `internal/decision/`

```
internal/decision/
  receipt.go    — DecisionReceipt struct and types
  logger.go     — Logger with channel-backed buffered writes
  query.go      — Query API backed by in-memory index
  redact.go     — Sensitive field redaction
```

Core types:

```go
type DecisionKind string

const (
    KindModelSelection    DecisionKind = "model_selection"
    KindFileSelection     DecisionKind = "file_selection"
    KindToolSelection     DecisionKind = "tool_selection"
    KindPlanStructure     DecisionKind = "plan_structure"
    KindContextBudget     DecisionKind = "context_budget"
    KindSelfHeal          DecisionKind = "self_heal"
    KindRetry             DecisionKind = "retry"
    KindVerification      DecisionKind = "verification"
    KindPhaseTransition   DecisionKind = "phase_transition"
    KindComplexity        DecisionKind = "complexity"
)

// DecisionReceipt is immutable after creation.
// All fields are unexported; access via getter methods.
type DecisionReceipt struct {
    id           string
    timestamp    time.Time
    phase        m31types.WorkflowPhase
    taskID       string
    kind         DecisionKind
    decision     string
    reason       string
    alternatives []string
    confidence   float64
    context      map[string]any
}

// Getters for all fields (omitted for brevity, follow standard pattern).
```

**Concurrency model:** The logger owns a goroutine that reads from a bounded channel (capacity 512). Writes to the channel are non-blocking: if the channel is full, the oldest receipt is dropped (ring buffer semantics). The goroutine batches writes: flush on phase transition or every 100ms, whichever comes first. Phase-transition flush is synchronous (blocks until durable) to guarantee decision completeness at phase boundaries. Within-phase flush is async (best-effort).

**Redaction:** Before writing, scan `decision`, `reason`, and `context` values for patterns matching API keys (`sk-*`, `ghp_*`, `glpat-*`, `xoxb-*`, `AKIA`), passwords (`password\s*=\s*"[^"]+"`), and private keys (`-----BEGIN`). Replace matched content with `[REDACTED]`. Reuse regex patterns from `common.go:SanitizeProviderError`.

**Persistence:** `.m31a/decisions.jsonl` (append-only, one JSON object per line). Rotate at 10MB, keep last 3 files. Same rotation pattern as existing log rotation.

**In-memory index:** On logger construction, build a `map[DecisionKind][]int` mapping kinds to byte offsets in the JSONL file. On startup, scan the file once to build the index. Queries (`QueryRecent`, `QueryByPhase`, `QueryByKind`) read from the in-memory index, not the file. The file is the durable store; the index is the query accelerator.

**Hook points in existing code:**

| Location | Decision | What to Log |
|----------|----------|-------------|
| `engine.go:modelForPhase()` | Which model tier won | Tier (per-phase override / agents config / default), model ID, reason |
| `execute.go:buildExecuteContext()` | Which files included | File path, relevance score, inclusion reason (task spec / code intel / dependency) |
| `tools/dispatcher.go:Execute()` | Tool selected | Tool name, input summary (first 200 chars), risk level |
| `plan.go:runPlan()` | Plan structure | Task count, dependency graph shape, research included |
| `execute.go:healTask()` | Self-heal trigger | Failure reason, heal strategy, attempt number |
| `engine.go:retryChatStream()` | Retry decision | Error class, retry count, backoff duration |
| `classify.go:ClassifyPrompt()` | Complexity level | Score breakdown, keyword matches, file count |
| `intent.go:ClassifyIntent()` | Intent classification | Intent type, confidence, LLM vs fallback |

**Implementation plan:**

1. Create `internal/decision/receipt.go` with immutable receipt type
2. Create `internal/decision/redact.go` with pattern-based redaction
3. Create `internal/decision/logger.go` with channel-backed buffered writer
4. Create `internal/decision/query.go` with in-memory index and query API
5. Extract `WorkflowState` from Engine (architecture prerequisite)
6. Add `state.decisionLog` to `WorkflowState`
7. Instrument `modelForPhase()` with decision logging
8. Instrument `buildExecuteContext()` with file selection logging
9. Instrument `dispatcher.Execute()` with tool selection logging
10. Instrument `healTask()` with heal decision logging
11. Add `/decisions` TUI screen (new screen constant, sub-model)
12. Add decision count badge to execute screen sidebar
13. Add `*slog.Logger` parameter to `decision.NewLogger()` constructor

**Risks:**
- Channel full drops receipts (mitigated: document durability contract — within-phase receipts may be lost on crash; phase-boundary receipts are durable)
- Performance overhead of logging (mitigated: channel write is ~100ns, non-blocking)
- TUI screen complexity (mitigated: simple list view with filter by kind)

**Testing strategy:**
- Unit: receipt serialization/deserialization roundtrip, redaction correctness, channel backpressure behavior
- Stress: `BenchmarkDecisionLoggerConcurrent` — 8 goroutines × 100 receipts, measure throughput, verify no corruption
- Integration: verify hook points produce receipts in a mock engine
- E2E: run a workflow, verify `.m31a/decisions.jsonl` contains expected decision kinds

**Expected user impact:** Users can run `/decisions` to see exactly why M31A made each choice. During autonomous execution, a sidebar badge shows decision activity. After completion, the decision log provides a complete audit trail. Sensitive data is redacted before persistence.

---

### Feature 2: Behavioral Verification Engine

**Problem.** `execute_quality.go` verifies tasks by checking `FILE contains STRING` and `FILE exists`. This cannot verify that code actually works—only that text exists. A task that adds a function passes verification even if the function has a syntax error, fails tests, or doesn't compile.

**Rationale.** Verification should prove that work succeeded. An engineer does not declare a task done by checking that a file contains certain text. They run the tests, build the project, and exercise the code. M31A should do the same.

**Architecture.**

Extend `internal/workflow/execute_quality.go` and create `internal/workflow/verify_runner.go`.

New criteria syntax:

```
# Existing (preserved)
FILE path/to/file contains "string"
FILE path/to/file exists
FILE path/to/file has "pattern"

# New: Command execution
CMD "go build ./..." exits 0
CMD "go test ./... -count=1" exits 0
CMD "npm run build" exits 0
CMD "python -m pytest" exits 0

# New: Output assertions
CMD "go test -v ./pkg/session/..." output contains "PASS"
CMD "git status --porcelain" output is empty

# New: HTTP verification (localhost only)
HTTP GET http://localhost:8080/api/health status 200
HTTP GET http://localhost:8080/api/health body contains "healthy"

# New: Negative assertions
FILE path/to/file not contains "debugger"
FILE path/to/file not contains "TODO"
FILE path/to/file not matches "password\s*=\s*\"[^\"]+\""

# New: Regex assertions
FILE path/to/file matches "func\s+Test\w+\("
FILE path/to/file matches "export\s+(default\s+)?function"
```

**Security model for CMD criteria:**

The LLM generates acceptance criteria during plan generation. Arbitrary command execution from LLM-generated criteria is a privilege escalation vector. The verification runner enforces:

1. **Command blocklist:** Identical to the Bash tool's blocklist (`rm -rf /`, fork bombs, `dd`, `mkfs`, `shutdown`, `curl | bash`, `eval`, backticks, `$(`). The blocklist is defined in `verify_runner.go` and shared with `bash.go` via a common function.

2. **Allowlist mode:** `VerifyRunner` accepts an `AllowedCommands []string` in its constructor. Only commands whose base name (first word) appears in the allowlist are executed. Default allowlist: `go`, `npm`, `yarn`, `pnpm`, `cargo`, `python`, `python3`, `pytest`, `node`, `make`, `git`, `curl`, `wget`. Configurable via `config.Verify.AllowedCommands`.

3. **Permission integration:** CMD criteria trigger the same permission modal as Bash tool calls. The user approves or denies each verification command.

4. **Input validation:** Criterion syntax is validated before execution. Malformed criteria (unclosed quotes, missing exit code, empty command) produce a parse error, not execution.

**Security model for HTTP criteria:**

1. **Localhost only:** HTTP verification is restricted to `localhost` and `127.0.0.1`. Requests to other hosts are rejected with a clear error message.

2. **No private IP access:** Reuse the `isPrivateIP` check from `webfetch.go` to block `169.254.x.x`, `10.x.x.x`, `192.168.x.x`, etc.

3. **No redirect following:** `CheckRedirect` returns an error for any redirect, preventing SSRF via redirect chains.

New types:

```go
type VerificationResult struct {
    Criterion  string `json:"criterion"`
    Passed     bool   `json:"passed"`
    Detail     string `json:"detail"`
    DurationMs int64  `json:"duration_ms"`
}

type VerifyRunner struct {
    workDir         string
    exec            func(ctx context.Context, cmd string, args ...string) *exec.Cmd
    allowedCommands []string
    logger          *slog.Logger
}

type VerifyRunnerOpts struct {
    AllowedCommands []string
    Logger          *slog.Logger
}
```

**Implementation plan:**

1. Create `internal/workflow/verify_runner.go` with `VerifyRunner`
2. Add command blocklist (shared with `bash.go` via `tools.CommandBlocked()` function)
3. Add `CMD` criterion parser with input validation (unclosed quote detection, empty command rejection)
4. Add `CMD` executor with permission modal integration, timeout (default 30s), output capture
5. Add `HTTP` criterion parser restricted to localhost
6. Add HTTP client with private IP blocking, no-redirect policy, timeout
7. Add negative assertion logic (`not contains`, `not matches`)
8. Add regex assertion logic (`matches` with compiled regex, pre-validated at parse time)
9. Add `output is empty` and `output contains` for command output
10. Refactor `verifyCriterion()` to dispatch to new runner for CMD/HTTP criteria
11. Add timeout per criterion (default 30s, configurable via `config.Verify.CriterionTimeout`)
12. Add criterion execution to decision log (Feature 1 integration)
13. Update coverage gates to recognize CMD/HTTP criteria in acceptance criteria
14. Add `FuzzCriterionParser` fuzz test

**Risks:**
- Command execution during verification could have side effects (mitigated: command blocklist + allowlist + permission modal)
- HTTP verification requires a running server (mitigated: skip with warning if connection refused, do not fail the criterion)
- Long-running commands could block verification (mitigated: per-criterion timeout, default 30s)
- Permission modal adds friction to verification (mitigated: users can auto-approve verification commands via config)

**Testing strategy:**
- Unit: criterion parser for all syntax variants, exit code matching, output matching, blocklist enforcement
- Fuzz: `FuzzCriterionParser` — random strings, verify no panics, no infinite loops
- Integration: run verification against a test project with known pass/fail criteria
- Security: attempt to inject `rm -rf /` as acceptance criterion, verify rejection
- E2E: create a task with CMD and HTTP criteria, verify correct pass/fail reporting

**Expected user impact:** Acceptance criteria become meaningful. A task can declare "CMD 'go test ./...' exits 0" and the system actually runs the tests. Negative assertions catch debug statements and TODO markers. HTTP criteria verify API endpoints respond correctly. Security is enforced without requiring the user to review every criterion.

---

### Feature 3: Adaptive Context Budgeting

**Problem.** Each task in the execute phase gets the full context—system prompt, project context, plan, code intelligence—regardless of task complexity. A one-line fix consumes the same context as a multi-file refactor. In long execution phases, later tasks hit context limits because earlier tasks consumed budget wastefully.

**Rationale.** Context window is the most valuable resource in autonomous execution. Proactive allocation across a plan prevents the "last 5 tasks fail" failure mode and enables more nuanced reasoning for complex tasks.

**Architecture.**

New package: `internal/contextbudget/`

```
internal/contextbudget/
  estimator.go   — Per-task token estimation with calibration
  allocator.go   — Budget allocation across plan
  tracker.go     — Runtime budget tracking with redistribution
```

Core types:

```go
type TaskBudget struct {
    TaskID       string
    EstimatedIn  int  // estimated input tokens
    EstimatedOut int  // estimated output tokens
    AllocatedIn  int  // actual allocation
    Priority     int  // 0=normal, 1=high (critical path)
}

type BudgetPlan struct {
    TotalContext    int              // model context window
    ReserveTokens   int              // system prompt + plan overhead
    TaskBudgets     []TaskBudget
    RemainingTokens int              // updated as tasks complete
}
```

**Estimation algorithm:**

1. **File complexity scan:** For each task's target files, use the codeintel indexer's existing `FileSymbols()` method to get symbol counts. Estimate: exported symbols * 30 tokens + imported symbols * 20 tokens.
2. **Task description complexity:** Word count * 1.3 (description tokens).
3. **Acceptance criteria complexity:** Count criteria * 100 tokens each.
4. **Calibration from execution data:** After each task completes, record actual input tokens (via `tokens.Estimator.EstimateMessages()`) vs estimated tokens. Maintain a running correction factor per file type (Go, TypeScript, Python, Rust). Apply correction factor to future estimates.
5. **Priority boost:** Tasks on the critical path (most dependents in the dependency graph) get 20% more budget.

**Allocation algorithm:**

1. Compute total available: `contextWindow - reserveTokens` (reserve = system prompt + plan + code intel = ~8K tokens).
2. Sum all task estimates (with calibration applied).
3. If total estimates < available: proportional allocation with 20% buffer each.
4. If total estimates > available: scale down proportionally, but enforce minimum 2K per task.
5. Track actual usage per task via token estimator.
6. Redistribute unused budget from completed tasks to remaining tasks.

**Integration with compaction:**

The existing `compaction.Compactor.ShouldCompact()` checks `tokenEst.EstimateMessages(messages) > contextLength - buffer`. The budget tracker replaces this check: compaction triggers when a task's actual usage exceeds 80% of its allocated budget, not at an absolute threshold. This prevents premature compaction (wasting context on summarization) and late compaction (overflow before compaction fires).

The `compactor` field on Engine remains. The budget tracker provides a `ShouldCompactForTask(taskID string, messages []Message) bool` method that the engine calls instead of `compactor.ShouldCompact()`.

**Integration points:**

- `engine.go:preflightContextCheck()` — use budget plan instead of ad-hoc truncation
- `execute.go:buildExecuteContext()` — truncate context to allocated budget per task
- `plan.go:runPlan()` — inject budget estimate into plan metadata for transparency
- `compaction` — defer to budget tracker's `ShouldCompactForTask()` method

**Implementation plan:**

1. Create `internal/contextbudget/estimator.go` with file/term/criteria estimation and calibration
2. Create `internal/contextbudget/allocator.go` with allocation algorithm
3. Create `internal/contextbudget/tracker.go` with runtime tracking and compaction integration
4. Add `state.budgetTracker` to `WorkflowState`
5. Compute budget plan at start of execute phase (after plan is finalized)
6. Add calibration recording to task runner (after each task completes, record actual vs estimated tokens)
7. Pass per-task budget to `buildExecuteContext()` for truncation
8. Update `preflightContextCheck()` to use budget-aware thresholds
9. Add budget status to decision log (Feature 1 integration)
10. Add budget indicator to TUI execute screen (progress: tokens used / allocated)
11. Add `*slog.Logger` parameter to constructors

**Risks:**
- Estimation accuracy may be low for unusual code (mitigated: calibration loop corrects over time; initial estimates use codeintel symbols as proxy)
- Budget reallocation overhead (mitigated: O(n) scan, n = remaining tasks)
- Interaction with auto-compaction (mitigated: budget tracker provides `ShouldCompactForTask()`, replacing absolute threshold)
- Calibration data not available for first session (mitigated: use conservative estimates, calibration improves from session 2 onward)

**Testing strategy:**
- Unit: estimation accuracy against known files, allocation algorithm edge cases (zero tasks, one task, all tasks over budget)
- Integration: run 10-task plan, verify budget redistribution works correctly after each task
- Calibration: run same 10-task plan twice, verify second run has better estimation accuracy
- E2E: run a 20-task plan that would overflow context without budgeting, verify all tasks complete

**Expected user impact:** Long execution phases (15+ tasks) complete reliably. Complex tasks get more context for better reasoning. Simple tasks use less context, leaving budget for harder work. Users see a budget indicator showing context consumption. Calibration improves accuracy over time.

---

### Feature 4: Persistent Project Knowledge

**Problem.** M31A starts every session with zero knowledge of the project's conventions, patterns, or history. It re-detects project type, re-analyzes architecture, and re-learns coding patterns from scratch. The ledger records session outcomes but is never queried during decision-making.

**Rationale.** An experienced engineer who has worked on a project for months knows its conventions, common failure modes, and architectural patterns without being told. M31A should build the same persistent understanding.

**Architecture.**

Knowledge accumulation lives as methods on `WorkflowState` in `internal/workflow/`. Persistence is a single JSON file at `.m31a/knowledge.json`. Extraction to a separate `internal/knowledge/` package is deferred to v1.6 after the data model is proven.

**Schema:**

```go
// Knowledge is the persistent project knowledge store.
// Version field enables future migration.
type Knowledge struct {
    Version      int                `json:"version"`
    Conventions  Conventions        `json:"conventions"`
    Patterns     []PatternEntry     `json:"patterns"`
    Failures     []FailureEntry     `json:"failures"`
    Architecture ArchitectureMap   `json:"architecture"`
}

type Conventions struct {
    NamingStyle   string   `json:"naming_style"`   // "camelCase", "snake_case", "PascalCase"
    ImportStyle   string   `json:"import_style"`   // "grouped", "flat"
    ErrorStyle    string   `json:"error_style"`     // "sentinel", "wrapping", "types"
    TestStyle     string   `json:"test_style"`      // "table_driven", "assertion", "behavior"
    PrefixPatterns []string `json:"prefix_patterns"` // ["Is", "Get", "New", "Has"]
    DetectedAt    time.Time `json:"detected_at"`
    Confidence    float64  `json:"confidence"`      // 0.0-1.0
}

type PatternEntry struct {
    Files       []string  `json:"files"`        // co-changed files
    Strategy    string    `json:"strategy"`     // edit strategy that succeeded
    Tools       []string  `json:"tools"`        // tool sequence (grep → edit)
    Count       int       `json:"count"`        // times observed
    LastSeen    time.Time `json:"last_seen"`
}

type FailureEntry struct {
    ErrorPattern string   `json:"error_pattern"` // regex or substring match
    FailureType  string   `json:"failure_type"`  // "tool_error", "test_failure", "build_error"
    FixStrategy  string   `json:"fix_strategy"`  // what healed it
    Files        []string `json:"files"`          // files involved
    Count        int      `json:"count"`
    LastSeen     time.Time `json:"last_seen"`
}

type ArchitectureMap struct {
    EntryPoints   []string          `json:"entry_points"`   // main.go, index.ts, app.py
    ConfigFiles   []string          `json:"config_files"`   // go.mod, package.json, etc.
    TestDirs      []string          `json:"test_dirs"`      // directories containing tests
    Subsystems    map[string][]string `json:"subsystems"`   // subsystem name → file paths
    BuiltAt       time.Time         `json:"built_at"`
}
```

**Convention detection:**

Conventions are detected by querying the codeintel indexer (after `Build()` completes) for exported identifiers. The knowledge module calls `idx.FileSymbols()` for each file, aggregates naming patterns, and classifies the dominant convention. This avoids duplicating parser logic.

Detection runs once per session (after codeintel build). If `.m31a/knowledge.json` exists and `Conventions.DetectedAt` is less than 24 hours old, skip detection and use cached conventions.

**Pattern memory:**

After each successful task in the execute phase, extract:
- List of files modified (from git diff)
- Edit strategy used (from decision log, if available)
- Tool sequence (from decision log, if available)

Update `Patterns` slice: if an entry with the same file set exists, increment `Count` and update `LastSeen`. Otherwise, append a new entry. Cap at 200 entries (FIFO eviction).

**Failure memory:**

After each successful self-heal, extract:
- Error pattern (from the failure string, extract the key error message)
- Failure type (from HealReport classfication)
- Fix strategy (from HealReport)
- Files involved (from HealReport)

Update `Failures` slice: if an entry with matching `ErrorPattern` exists, increment `Count` and update `LastSeen`. Otherwise, append. Cap at 100 entries (FIFO eviction).

**Architecture map:**

Built lazily (first query triggers build). Uses codeintel's existing file listing and import graph. Entry points: files matching `main.go`, `index.ts`, `app.py`, `main.rs`. Config files: `go.mod`, `package.json`, `Cargo.toml`, `requirements.txt`, `pyproject.toml`. Test dirs: directories containing test files. Subsystems: top-level directories (excluding `vendor`, `node_modules`, `.git`, `.m31a`).

**Integration points:**

- `engine.go:buildSystemPrompt()` — inject `Conventions` summary (naming style, import style) into system prompt
- `plan.go:buildPlanContext()` — inject `ArchitectureMap` summary into plan context
- `execute.go:buildExecuteContext()` — inject relevant `Failures` for current task's files
- `execute.go:healTask()` — query `Failures` for similar error patterns to inform heal strategy
- `ledger` — cross-reference `Patterns` with session outcomes

**Implementation plan:**

1. Define `Knowledge` schema in `internal/types/types.go` (with `Version` field)
2. Create `internal/workflow/knowledge_io.go` with JSON read/write and migration
3. Create `internal/workflow/knowledge_conventions.go` with codeintel-based detection
4. Create `internal/workflow/knowledge_patterns.go` with co-change tracking
5. Create `internal/workflow/knowledge_failures.go` with failure → fix mapping
6. Add `state.knowledge` to `WorkflowState`
7. Load knowledge at session start (if file exists, read and validate version)
8. Run convention detection after codeintel build (skip if recent)
9. Update patterns after each successful task in Execute phase
10. Update failures after each self-heal
11. Inject knowledge into `buildSystemPrompt()`, `buildPlanContext()`, `buildExecuteContext()`
12. Add `.m31a/knowledge.json` to `.gitignore` template in `initialize.go`
13. Add `/knowledge` TUI screen (read-only view of stored knowledge)
14. Add `*slog.Logger` parameter to knowledge functions

**Risks:**
- Knowledge file growth (mitigated: pattern cap 200, failure cap 100, FIFO eviction)
- Incorrect convention detection (mitigated: confidence scoring, skip if < 10 files parsed)
- Knowledge files contain code patterns (mitigated: project-local only, added to `.gitignore`)
- Convention detection depends on codeintel build (mitigated: knowledge loads regardless; conventions are optional)

**Testing strategy:**
- Unit: convention detection against known project structures, pattern extraction from mock edits, schema roundtrip
- Failure injection: truncated JSON, unknown fields, version mismatch, permission denied, disk full
- Integration: run 5-session workflow, verify knowledge accumulates correctly
- E2E: verify that knowledge from session 1 improves planning in session 2 (conventions injected into system prompt)

**Expected user impact:** After a few sessions, M31A knows the project's conventions and avoids suggesting style-inconsistent code. It remembers what failed before and avoids repeating mistakes. It knows which files are related and includes them proactively. Knowledge is transparent via `/knowledge`.

---

### Feature 5: Selective Rollback (File-Level)

**Problem.** `pkg/rollback/` operates on entire commits. When self-healing produces a partially correct fix, the system cannot surgically undo the broken parts while keeping the working parts. The only option is to revert the entire commit or manually edit files.

**Rationale.** An experienced engineer does not throw away an entire commit because one file has a bug. They fix the specific file. Selective rollback enables surgical correction of autonomous mistakes without discarding good work.

**Scope for v1.5:** File-level rollback only. Hunk-level rollback is deferred to v1.6 due to the complexity of `git apply --reverse` with conflict handling.

**Architecture.**

Extend `pkg/rollback/rollback.go`:

```go
// ChangedFiles returns the list of files changed in a commit.
func (r *Rollback) ChangedFiles(ctx context.Context, commit string) ([]string, error)

// FileDiff returns the diff for a specific file in a commit.
func (r *Rollback) FileDiff(ctx context.Context, commit, file string) (string, error)

// RevertFiles reverts specific files to their state in the given commit.
// Stashes uncommitted changes before revert, restores stash after.
func (r *Rollback) RevertFiles(ctx context.Context, commit string, files []string) (*RollbackResult, error)
```

`RevertFiles` uses `git checkout <commit> -- <file>` for each file. Before checkout, it stashes uncommitted changes via the existing `stashIfDirty()` pattern. After checkout, it pops the stash. If stash pop fails, the result reports `ChangesStashed: true` so the user can recover manually.

**Integration points:**

- Self-heal: after heal produces a bad edit in one file, revert only that file and retry
- Verify phase: when verification fails for a specific file, revert that file
- TUI: `/rollback file` screen with file picker showing changed files in last commit
- Decision log: record selective rollback decisions

**Implementation plan:**

1. Add `ChangedFiles()` method to `pkg/rollback/rollback.go`
2. Add `FileDiff()` method
3. Add `RevertFiles()` method using `git checkout` with stash-if-dirty
4. Add `/rollback file` TUI screen with file list and diff preview
5. Integrate with self-heal: after heal, if verify fails for a specific file, revert that file
6. Add rollback decisions to decision log
7. Document deferred hunk-level rollback in code comments

**Risks:**
- Partial revert could break dependencies between files (mitigated: warn when reverting files that are imported by other changed files, using codeintel import graph)
- Stash pop after revert could fail (mitigated: report stash state in result, user recovers manually)

**Testing strategy:**
- Unit: ChangedFiles, FileDiff, RevertFiles against test repos
- Integration: create a commit with 5 files, revert 2, verify only those 2 changed
- Edge case: revert file with uncommitted changes, verify stash preserved
- E2E: self-heal produces a bad edit in one file, selective rollback reverts it, verify passes

**Expected user impact:** Self-healing becomes more precise. Instead of reverting an entire commit when one file is wrong, the system surgically fixes only the broken file. Users can browse changed files and selectively undo specific changes via `/rollback file`.

---

### Feature 6: Checkpoint Resume (Single-Session)

**Problem.** Checkpoints exist but cannot be browsed or resumed at specific tasks. Interruption produces no summary. Context is lost between sessions.

**Rationale.** An engineer working with M31A for eight hours needs continuity. They need to resume where they left off and recover from interruptions gracefully.

**Scope for v1.5:** Single-session checkpoint resume with interruption summary. Multi-session storage is deferred to v1.6 (requires session manager rewrite, not a modification).

**Architecture.**

**Extended checkpoint data:**

```go
type Checkpoint struct {
    SessionID      string              `json:"session_id"`
    Phase          m31types.WorkflowPhase `json:"phase"`
    TaskIndex      int                 `json:"task_index"`
    TaskID         string              `json:"task_id"`
    Goal           string              `json:"goal"`
    PlanMarkdown   string              `json:"plan_markdown"`
    CompletedTasks []string            `json:"completed_tasks"`
    Timestamp      time.Time           `json:"timestamp"`
}
```

**Interruption summary (template-based, no LLM call):**

When the workflow is cancelled (SIGINT, user cancel), generate a summary from structured data. No LLM call required — the data is already captured by task list, decision log, and git log.

```
## Session Summary — 2026-06-28 14:30

**Goal:** Add JWT authentication to API endpoints

**Completed:**
- Task 1: Create auth middleware (commit abc1234)
- Task 2: Add token validation (commit def5678)

**In Progress:**
- Task 3: Add refresh token support (50% complete, files modified: auth/refresh.go)

**Remaining:**
- Task 4: Add integration tests
- Task 5: Update API documentation

**Decisions:** 12 logged (run /decisions to view)
```

Store in `.m31a/summary.md` (overwritten each time; only the latest interruption summary is kept).

**Force checkpoint before summary:** Before generating the summary, force a checkpoint save via `e.sessionMgr.SaveCheckpoint()`. This ensures the summary reflects the latest state, not a stale checkpoint from 5 minutes ago.

**Checkpoint-based resume:**

`/resume` screen shows available checkpoints with phase, task progress, and timestamp. User selects a checkpoint and execution resumes from that task's index in the task list.

**Context carryover:**

When resuming from a checkpoint, inject into the system prompt:

```
## Previous Session Context

**Goal:** [from checkpoint]
**Phase:** [from checkpoint]
**Completed Tasks:** [from checkpoint]
**Recent Decisions:** [last 10 from decisions.jsonl, if available]
```

This is a subset of the decision log and task state, not raw messages. It provides continuity without injecting stale conversation.

**Implementation plan:**

1. Extend checkpoint data with `TaskIndex`, `CompletedTasks`, `Goal`
2. Force checkpoint save before summary generation
3. Generate interruption summary from structured data (template, no LLM)
4. Store summary in `.m31a/summary.md`
5. Add context carryover logic in `engine.go:buildSystemPrompt()`
6. Add `/resume` checkpoint selection screen (show available checkpoints with metadata)
7. Add checkpoint validation on load (verify session ID, phase, task list integrity)

**Risks:**
- Summary may be stale if checkpoint was saved long ago (mitigated: force checkpoint save before summary)
- Context carryover may inject stale information (mitigated: only import decisions and task state, not raw messages)
- Checkpoint resume complexity (mitigated: validate checkpoint integrity on load, reject corrupted checkpoints)

**Testing strategy:**
- Unit: checkpoint serialization/deserialization, summary generation from mock data
- Integration: run workflow, cancel at task 5, verify summary generated with correct state
- E2E: run workflow, cancel at task 5, resume from checkpoint 5, verify continuity
- Edge case: corrupt checkpoint file, verify graceful rejection

**Expected user impact:** Interruption produces a clear summary. Resume picks up exactly where they left off. Context carries over between sessions.

---

### Feature 7: Enhanced Self-Healing with Explanation

**Problem.** Self-healing in `execute.go:healTask()` retries failed tasks but does not explain what failed, what was tried, or what worked. The user sees "task completed" or "task failed" with no insight into the repair process.

**Rationale.** When an engineer fixes a bug, they explain what was wrong and how they fixed it. M31A should do the same. This builds trust and feeds into project knowledge (Feature 4).

**Architecture.**

Define `HealReport` in `internal/types/types.go`:

```go
type HealReport struct {
    TaskID        string        `json:"task_id"`
    Attempt       int           `json:"attempt"`
    FailureReason string        `json:"failure_reason"`
    FailureType   string        `json:"failure_type"` // "tool_error", "test_failure", "build_error", "quality_gate"
    Strategy      string        `json:"strategy"`     // "retry_with_context", "file_rollback", "alternative_approach"
    FilesChanged  []string      `json:"files_changed"`
    Success       bool          `json:"success"`
    Duration      time.Duration `json:"duration"`
}
```

**Data currently captured by `healTask()`:**

- Task metadata (ID, action, description, files, acceptance criteria)
- Failure string (free-form)
- Current file state (via `readTaskFiles`)
- Git diff stat (last 3 commits, `--stat` only)
- Code intelligence context (via `FormatContext`)

**Data added by this feature:**

- Failure classification (rule-based: check error string for test/build/tool patterns)
- Heal strategy description (what context was added, what files were modified)
- Files changed during heal (from git diff after heal commit)

After each heal attempt:
1. Classify the failure (tool error vs test failure vs build error vs quality gate)
2. Record the heal strategy (what context was added, what files were modified)
3. If successful, extract the fix pattern and store in knowledge (Feature 4)
4. If all attempts exhausted, produce a failure report with recommendations
5. Log all heal decisions to decision log (Feature 1)
6. Write HealReport to decision log (not accumulated on Engine)

**Failure classification rules:**

```go
func classifyFailure(output string) string {
    if strings.Contains(output, "FAIL") || strings.Contains(output, "test") {
        return "test_failure"
    }
    if strings.Contains(output, "build") || strings.Contains(output, "compile") {
        return "build_error"
    }
    if strings.Contains(output, "timeout") || strings.Contains(output, "deadline") {
        return "timeout"
    }
    return "tool_error"
}
```

**Integration points:**

- `execute.go:healTask()` — build HealReport during execution, write to decision log
- `verify.go:tryBisectHeal()` — build HealReport for bisect-triggered heals
- `knowledge` (Feature 4) — after successful heal, update failure memory
- `decision/logger.go` (Feature 1) — log heal decisions
- TUI: display heal progress as a notification toast

**Implementation plan:**

1. Add `HealReport` type to `internal/types/types.go`
2. Add `classifyFailure()` helper to `internal/workflow/execute.go`
3. Modify `healTask()` to build HealReport during execution
4. Modify `tryBisectHeal()` to build HealReport
5. Write HealReport to decision log (via `state.decisionLog`)
6. After successful heal, update knowledge failure memory (via `state.knowledge`)
7. Add heal notification toast to TUI
8. Add heal history to `/decisions` screen (filter by KindSelfHeal)

**Risks:**
- Heal report generation adds overhead (mitigated: lightweight struct, written to decision log's existing channel)
- Incorrect failure classification (mitigated: rule-based classifier with fallback to "tool_error"; easy to extend)

**Testing strategy:**
- Unit: `classifyFailure()` against known error strings, HealReport serialization roundtrip
- Integration: trigger heal with known failure, verify report accuracy
- E2E: run workflow with intentionally broken task, verify heal report is generated and logged

**Expected user impact:** When self-healing occurs, users see exactly what went wrong, what was tried, and what worked. The heal history builds a knowledge base of failure modes that improves future execution.

---

## Milestones

### M1a: Decision Transparency

**Scope:** Feature 1 (decision logging)

**Rationale:** Trust foundation. Before improving intelligence, make the system explainable. Users need to understand what M31A is doing before they can trust it to do more.

**Deliverables:**
- `internal/decision/` package with receipt types, logger, query API, redaction
- `WorkflowState` extraction from Engine
- Decision logging hooks in engine, dispatcher, execute, plan, classify, intent
- `/decisions` TUI screen

**Dependencies:** None (builds on existing infrastructure)

**Estimated effort:** 2-3 weeks

---

### M1b: Selective Rollback

**Scope:** Feature 5 (file-level rollback)

**Rationale:** Enables surgical correction. Independent of decision transparency; can be built in parallel.

**Deliverables:**
- `ChangedFiles()`, `FileDiff()`, `RevertFiles()` on `pkg/rollback/`
- `/rollback file` TUI screen
- Integration with self-heal (revert specific file on verify failure)

**Dependencies:** None (builds on existing `pkg/rollback/`)

**Estimated effort:** 1-2 weeks

---

### M2: Self-Heal Explanation + Behavioral Verification

**Scope:** Features 2, 7

**Rationale:** Quality foundation. After making decisions explainable (M1a), make verification meaningful and self-healing transparent. These features share the verification runner and decision log integration.

**Deliverables:**
- `HealReport` type and classification logic
- `internal/workflow/verify_runner.go` with CMD, HTTP, negative, regex criteria
- Command blocklist (shared with Bash tool)
- Permission modal integration for verification commands
- Refactored `execute_quality.go` dispatching to new runner
- Criterion timeout enforcement
- Integration with decision log (M1a)
- Updated coverage gates to recognize new criteria syntax

**Dependencies:** M1a (decision log integration)

**Estimated effort:** 3-4 weeks

---

### M3: Adaptive Context + Project Knowledge

**Scope:** Features 3, 4

**Rationale:** Intelligence foundation. After trust and verification are solid, improve the system's ability to reason about code and learn from experience.

**Deliverables:**
- `internal/contextbudget/` package with estimator, allocator, tracker
- Budget computation at execute phase start
- Per-task context truncation based on budget
- Calibration loop (actual vs estimated tokens per task)
- Knowledge schema in `internal/types/types.go`
- `internal/workflow/knowledge_*.go` with I/O, conventions, patterns, failures
- Convention detection via codeintel
- Pattern and failure memory updated during execution
- Knowledge injection into system prompt and plan context
- `.m31a/knowledge.json` in `.gitignore`
- `/knowledge` TUI screen

**Dependencies:** M1a (decision log feeds knowledge), M2 (verification results feed knowledge)

**Estimated effort:** 4-5 weeks

---

### M4: Checkpoint Resume + Interruption Summary

**Scope:** Feature 6 (single-session checkpoint resume)

**Rationale:** Session foundation. After intelligence and trust are established, enable reliable interruption recovery.

**Deliverables:**
- Extended checkpoint data with task index and completed tasks
- Interruption summary generation (template-based, no LLM)
- Force checkpoint before summary
- Context carryover from checkpoint
- `/resume` checkpoint selection screen
- Checkpoint validation on load

**Dependencies:** M1a (decisions feed summaries)

**Estimated effort:** 2-3 weeks

---

## Features Explicitly Rejected

| Feature | Why Rejected |
|---------|-------------|
| Browser automation | Outside M31A's scope as terminal-native agent. Massive complexity for marginal benefit. |
| Editor integrations (VS Code, JetBrains) | M31A is terminal-native. Multiple integration APIs multiply maintenance burden without improving autonomy. |
| MCP server | External protocol dependency for a different integration model. M31A's tool system is its integration point. |
| Plugin ecosystem | Requires stable APIs, versioning, dependency management, security sandboxing. This is a product in itself. Skills provide extensibility without ecosystem overhead. |
| Multi-model concurrent execution | Running multiple models simultaneously for comparison adds complexity without improving autonomy. The existing model selection with fallback is sufficient. |
| Natural language database queries | Niche feature that diverts from core autonomous engineering. The bash tool already handles database interactions. |
| Collaborative multi-user sessions | M31A is a single-user terminal tool. Multi-user adds session locking, conflict resolution, and permission complexity that dilutes focus. |
| Auto-generated PR descriptions | The ship phase already generates changelogs. PR description generation is an editor integration concern. |
| Code review by AI | M31A's verify phase already checks code quality. Adding a separate review step adds latency without improving the verification that already exists. |
| Real-time code collaboration | M31A works on local files. Real-time collaboration requires a server infrastructure that is outside the terminal-native scope. |
| Hunk-level rollback (v1.5) | Deferred to v1.6. `git apply --reverse` with conflict handling is a research problem. File-level rollback covers the primary use case. |
| Multi-session storage (v1.5) | Deferred to v1.6. Requires session manager rewrite (path resolution, locking, concurrency), not a modification. |
| `/sessions` browsing screen (v1.5) | Deferred to v1.6. Users rarely browse old sessions. Resume and summary are the high-value features. |

---

## Success Metrics

| Metric | Baseline (v1.4) | Target (v1.5) | Measurement |
|--------|-----------------|---------------|-------------|
| Decision explainability | 0% of decisions logged | 100% of major decisions logged | Count decision types in JSONL per session |
| Verification accuracy | Text-substring only | Behavioral verification (CMD, HTTP, negative) | Count criterion types used per session |
| Task completion rate (15+ task plans) | ~60% (context overflow) | ~90% (budget-managed) | Count completed vs total tasks in long plans |
| Self-heal success rate | ~50% (no explanation) | ~70% (with explanation and knowledge) | Count successful heals / total heal attempts |
| Session resume success | N/A (no resume) | 90% successful resume from checkpoint | Count successful resumes / total resume attempts |
| Knowledge accuracy | 0% (no knowledge) | 80% convention detection accuracy | Manual review of detected conventions vs actual |
| User trust (qualitative) | "Black box" feedback | "Explainable partner" feedback | User surveys, issue reports |
| Unnecessary edits | ~30% of edits are revised | ~15% of edits are revised | Count edits that trigger self-heal / total edits |
| Context overflow failures | ~20% of long sessions | ~5% of long sessions | Count context overflow errors / total sessions |
| Time to first correct edit | 2.5 attempts average | 1.8 attempts average | Count tool calls per task / total tasks |

---

## Migration Requirements

### v1.4 → v1.5 Session Data

v1.5 must handle v1.4 data gracefully:

1. **Missing directories:** Create `.m31a/` subdirectories (`knowledge/`, `sessions/`) on first run if they don't exist.
2. **Existing session:** Import existing `session.json` + `messages.json` as the current session. Do not modify v1.4-format files.
3. **Missing files:** `decisions.jsonl` does not exist until first decision is logged. `knowledge.json` does not exist until first knowledge is written. `summary.md` does not exist until first interruption.
4. **Forward compatibility:** v1.4 can read v1.5 session data (it ignores unknown fields in JSON). v1.5 can read v1.4 session data (it treats missing fields as zero values).

### Config Migration

v1.5 adds new config fields:

```toml
[verify]
allowed_commands = ["go", "npm", "yarn", "pnpm", "cargo", "python", "python3", "pytest", "node", "make", "git"]
criterion_timeout = "30s"

[knowledge]
max_patterns = 200
max_failures = 100
convention_ttl = "24h"

[decision_log]
buffer_size = 512
flush_interval = "100ms"
rotate_at_mb = 10
max_files = 3

[contextbudget]
enabled = true
compaction_threshold = 0.8
```

New fields have defaults. v1.4 config files load correctly in v1.5 (new fields get defaults). No schema versioning required for v1.5 (add `config_version` field in v1.6 when multi-session is added).

---

## Observability Requirements

### Structured Logging

Each new subsystem accepts `*slog.Logger` in its constructor:

- `decision.NewLogger(path string, opts LoggerOpts, logger *slog.Logger)`
- `contextbudget.NewTracker(opts TrackerOpts, logger *slog.Logger)`
- Knowledge functions accept `logger *slog.Logger` as first parameter

Debug-level logging for:

- Decision receipts written (kind, task ID)
- Budget allocations computed (task ID, estimated, allocated)
- Knowledge entries loaded (convention confidence, pattern count, failure count)
- Rollback operations performed (files reverted, commit hash)
- Verification criteria executed (criterion, passed, duration)

### Metrics Extension

Extend `internal/metrics/Collector` with:

```go
func (c *Collector) RecordDecision(kind string)
func (c *Collector) RecordVerification(criterionType string, passed bool)
func (c *Collector) RecordBudgetAccuracy(estimated, actual int)
func (c *Collector) RecordKnowledgeHit(kind string)
func (c *Collector) RecordHealOutcome(failureType string, success bool)
```

These enable the success metrics to be measured objectively via `/metrics`.

### Debug Commands

- `/decisions` — browse decision log with filter by kind
- `/knowledge` — view stored knowledge (conventions, patterns, failures, architecture)
- `/resume` — list available checkpoints with metadata
- `/metrics` — view extended metrics including new subsystem metrics

---

## Architecture Impact Summary

| Area | Change | Files Modified | Files Created |
|------|--------|----------------|---------------|
| Engine decomposition | Extract WorkflowState | `engine.go`, all phase files | (none) |
| Decision logging | New package + engine hooks | `engine.go`, `execute.go`, `plan.go`, `classify.go`, `intent.go`, `tools/dispatcher.go` | `internal/decision/receipt.go`, `logger.go`, `query.go`, `redact.go` |
| Behavioral verification | Extend execute_quality | `execute_quality.go`, `coverage_gates.go`, `tools/bash.go` | `internal/workflow/verify_runner.go` |
| Context budgeting | New package + engine integration | `engine.go`, `execute.go`, `plan.go` | `internal/contextbudget/estimator.go`, `allocator.go`, `tracker.go` |
| Project knowledge | Methods on WorkflowState + persistence | `engine.go`, `execute.go`, `plan.go`, `internal/types/types.go` | `internal/workflow/knowledge_io.go`, `knowledge_conventions.go`, `knowledge_patterns.go`, `knowledge_failures.go` |
| Selective rollback | Extend rollback package | `pkg/rollback/rollback.go` | (none) |
| Checkpoint resume | Extend session/checkpoint | `engine.go`, `pkg/session/manager.go` | (none) |
| Self-healing explanation | Extend execute + types | `internal/workflow/execute.go`, `verify.go`, `internal/types/types.go` | (none) |
| TUI screens | New screens | `internal/tui/app_state.go`, `app_update.go` | `internal/tui/decisions.go`, `knowledge.go`, `resume.go`, `rollback_file.go` |
| Metrics | Extend collector | `internal/metrics/collector.go` | (none) |

---

## Implementation Priority

```
Week 1-3:   M1a (Decision Transparency) + M1b (Selective Rollback) — parallel workstreams
Week 4-7:   M2 (Self-Heal Explanation + Behavioral Verification)
Week 8-12:  M3 (Adaptive Context + Project Knowledge)
Week 13-15: M4 (Checkpoint Resume + Interruption Summary)
Week 16-17: Integration testing, migration testing, documentation
Week 18-19: Release candidate, bug fixes, stress testing
Week 20:    v1.5.0 release
```

---

## What M31A Becomes

v1.4 is a workflow executor. It runs phases and produces commits.

v1.5 is a thinking partner. It explains its reasoning, proves its work is correct, learns from every session, and maintains continuity across hours of autonomous operation.

The difference is not features. It is intelligence made visible.
