# v8.15 Behavior Loop Runtime Policy Auto-Tuning Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Build a mature behavior loop where corrections, user preferences, and recall outcomes automatically tune runtime memory policies without requiring manual operator updates.

**Architecture:** Add a bounded policy-learning layer that consumes correction/preference signals, computes safe parameter adjustments, and applies them at runtime with auditability and rollback. Keep all updates constrained by explicit envelopes so auto-tuning cannot violate existing retrieval/config invariants.

**Tech Stack:** TypeScript, orchestrator, recall policy engine, storage state logs, node:test.

---

### Task 1: Define policy-learning state model and safety envelope

**Files:**
- Modify: `src/types.ts`
- Modify: `src/schemas.ts`
- Modify: `src/config.ts`
- Modify: `openclaw.plugin.json`
- Test: `tests/config-proactive-policy.test.ts`

**Step 1: Write failing config/schema tests**
- Add new config keys:
  - `behaviorLoopAutoTuneEnabled` (default `false`)
  - `behaviorLoopLearningWindowDays` (default `14`)
  - `behaviorLoopMinSignalCount` (default `10`)
  - `behaviorLoopMaxDeltaPerCycle` (default `0.1`)
  - `behaviorLoopProtectedParams` (default includes strict invariants)
- Assert numeric `0` behavior remains explicit and never coerced.

**Step 2: Run tests to verify failure**
- Run: `npx tsx --test tests/config-proactive-policy.test.ts`
- Expected: FAIL.

**Step 3: Implement config + type model**
- Add typed `BehaviorLoopPolicyState` and `BehaviorLoopAdjustment` contracts.
- Add safe defaults and parsing guards.

**Step 4: Re-run tests**
- Run: `npx tsx --test tests/config-proactive-policy.test.ts`
- Expected: PASS.

**Step 5: Commit**
- `git add src/types.ts src/schemas.ts src/config.ts openclaw.plugin.json tests/config-proactive-policy.test.ts`
- `git commit -m "feat: add behavior loop auto-tuning config and state contracts"`

### Task 2: Build correction/preference signal ingestion pipeline

**Files:**
- Create: `src/behavior-signals.ts`
- Modify: `src/storage.ts`
- Modify: `src/orchestrator.ts`
- Test: `tests/behavior-signals.test.ts`
- Test: `tests/storage-policy-state.test.ts`

**Step 1: Write failing signal-ingestion tests**
- Cases:
  - Corrections generate negative/override signals.
  - Confirmed preferences generate positive routing/ranking signals.
  - Duplicate corrections dedupe by memory id + hash.
  - Signals are namespace-safe and timestamped.

**Step 2: Run tests to verify failure**
- Run: `npx tsx --test tests/behavior-signals.test.ts tests/storage-policy-state.test.ts`
- Expected: FAIL.

**Step 3: Implement signal extraction + persistence**
- Add append-only ledger: `state/behavior-signals.jsonl`.
- Wire extraction/correction paths to emit normalized signals.
- Keep writes fail-open and non-blocking.

**Step 4: Re-run tests**
- Run: `npx tsx --test tests/behavior-signals.test.ts tests/storage-policy-state.test.ts`
- Expected: PASS.

**Step 5: Commit**
- `git add src/behavior-signals.ts src/storage.ts src/orchestrator.ts tests/behavior-signals.test.ts tests/storage-policy-state.test.ts`
- `git commit -m "feat: ingest corrections and preferences into behavior signal ledger"`

### Task 3: Implement bounded policy learner

**Files:**
- Create: `src/behavior-learner.ts`
- Modify: `src/lifecycle.ts`
- Modify: `src/recall-query-policy.ts`
- Test: `tests/behavior-learner.test.ts`

**Step 1: Write failing learner tests**
- Verify learner can adjust only allowed parameters (examples):
  - retrieval budget mode preference in ambiguous prompts
  - demotion/promotion sensitivity weights
  - recency/access boosting coefficients
- Verify protected params remain immutable.
- Verify max delta per cycle is enforced.

**Step 2: Run tests to verify failure**
- Run: `npx tsx --test tests/behavior-learner.test.ts`
- Expected: FAIL.

**Step 3: Implement learner**
- Compute rolling aggregates over `behavior-signals.jsonl` within configured window.
- Produce proposed adjustments with confidence + evidence counts.
- Reject low-signal updates below `behaviorLoopMinSignalCount`.

**Step 4: Re-run tests**
- Run: `npx tsx --test tests/behavior-learner.test.ts`
- Expected: PASS.

**Step 5: Commit**
- `git add src/behavior-learner.ts src/lifecycle.ts src/recall-query-policy.ts tests/behavior-learner.test.ts`
- `git commit -m "feat: add bounded behavior learner for runtime policy updates"`

### Task 4: Runtime policy application + rollback path

**Files:**
- Create: `src/policy-runtime.ts`
- Modify: `src/orchestrator.ts`
- Modify: `src/retrieval.ts`
- Test: `tests/policy-runtime-application.test.ts`

**Step 1: Write failing runtime tests**
- Ensure learned policy state is loaded and applied at runtime.
- Ensure `no_recall` / `minimal` / `full` / `graph_mode` contracts remain reachable.
- Ensure rollback restores prior policy snapshot on invalid update.

**Step 2: Run tests to verify failure**
- Run: `npx tsx --test tests/policy-runtime-application.test.ts`
- Expected: FAIL.

**Step 3: Implement runtime policy manager**
- Add atomic snapshot files:
  - `state/policy-runtime.json`
  - `state/policy-runtime.prev.json`
- Apply adjustments only after schema + invariant validation.
- Fail-open to baseline config when runtime policy is absent/corrupt.

**Step 4: Re-run tests**
- Run: `npx tsx --test tests/policy-runtime-application.test.ts`
- Expected: PASS.

**Step 5: Commit**
- `git add src/policy-runtime.ts src/orchestrator.ts src/retrieval.ts tests/policy-runtime-application.test.ts`
- `git commit -m "feat: add runtime policy application with guarded rollback"`

### Task 5: Add explainability and operator observability

**Files:**
- Modify: `src/cli.ts`
- Modify: `src/recall-state.ts`
- Test: `tests/cli-policy-tuning.test.ts`
- Docs: `docs/operations.md`

**Step 1: Write failing CLI tests**
- Add commands:
  - `openclaw engram policy-status`
  - `openclaw engram policy-diff --since 7d`
  - `openclaw engram policy-rollback`
- Validate output includes parameter deltas + evidence counts.

**Step 2: Run tests to verify failure**
- Run: `npx tsx --test tests/cli-policy-tuning.test.ts`
- Expected: FAIL.

**Step 3: Implement CLI + telemetry surfaces**
- Record per-turn policy version in recall telemetry.
- Expose top contributing signals for each adjustment.

**Step 4: Re-run tests**
- Run: `npx tsx --test tests/cli-policy-tuning.test.ts`
- Expected: PASS.

**Step 5: Commit**
- `git add src/cli.ts src/recall-state.ts tests/cli-policy-tuning.test.ts docs/operations.md`
- `git commit -m "feat: add behavior-loop policy observability and rollback CLI"`

### Task 6: Guardrails and hardening validation

**Files:**
- Modify: `docs/ops/pr-review-hardening-playbook.md`
- Modify: `docs/setup-config-tuning.md`
- Modify: `CHANGELOG.md`

**Step 1: Add behavior-loop hardening checklist**
- Must verify:
  - artifact isolation preserved
  - cap-after-filter ordering preserved
  - config `enabled=false` and `0` semantics preserved
  - planner mode semantics unchanged

**Step 2: Run full verification**
- Run:
  - `npm run check-types`
  - `npm test`
  - `npm run build`
  - `npm run preflight:quick`
- Expected: all PASS.

**Step 3: Commit**
- `git add docs/ops/pr-review-hardening-playbook.md docs/setup-config-tuning.md CHANGELOG.md`
- `git commit -m "docs: add hardening and rollout guidance for behavior-loop auto-tuning"`

## Rollout Notes

1. Ship with `behaviorLoopAutoTuneEnabled=false`.
2. Enable in canary namespace only.
3. Require minimum signal volume before first adjustment.
4. Monitor policy diff stability and rollback rate before wider enablement.
