# v8.3 PR20: Lifecycle Memory Policy Engine (MemoryOS-Inspired, Engram-Native)

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

## Objective

Implement a deterministic, policy-driven lifecycle engine that governs **promotion, decay, revalidation, and retrieval eligibility** for memories using existing Engram signals (access, recency, confidence, feedback, contradiction status).

This adds the most transferable MemoryOS primitive (lifecycle promotion logic) without replacing Engram’s current architecture.

## Rationale (linked to BM-493923)

BM-493923 found that MemoryOS’s key advantage is not storage format but a coherent lifecycle loop (short→mid→long with heat-triggered promotion). Engram already has stronger operations/safety and already ships:

- memory statuses (`active/superseded/archived`)
- access tracking (`accessCount`, `lastAccessed`)
- confidence tiers + TTL
- consolidation/archival passes
- feedback and negative-example biasing

Current gap: no unified lifecycle score/state machine spanning these signals. Lifecycle decisions are split across ad-hoc rules (speculative expiry, commitment decay, low-importance archival). PR20 closes that gap with an explicit policy layer.

---

## Scope

1. Add lifecycle metadata + state machine fields to memory frontmatter and/or lifecycle state index.
2. Add lifecycle scoring pass in consolidation (`runConsolidation`) with deterministic promotion/demotion rules.
3. Add retrieval filtering/weighting by lifecycle state (fail-open, configurable).
4. Add observability: lifecycle metrics file and debug tool output.
5. Keep migration non-breaking for existing memory files.

## Non-Scope

- No cross-system schema unification with Atlas/bookmark registries in this PR.
- No mandatory citation enforcement in assistant response generation (future PR).
- No expensive LLM-in-the-loop policy adjudication by default.
- No destructive data migration (no bulk deletes/rewrites required).

---

## PR Slice Plan (Recommended)

Deliver v8.3 in four smaller PRs, all default-off via config flags:

1. **PR 20A — Data model + config surface**
- Add lifecycle frontmatter fields as optional.
- Add config flags and parsing with strict backward compatibility.
- No lifecycle decisions or retrieval changes yet.

2. **PR 20B — Deterministic lifecycle engine module**
- Add `src/lifecycle.ts` and pure scoring/transition functions.
- Unit-test deterministic behavior and edge cases.
- No orchestrator wiring yet.

3. **PR 20C — Consolidation wiring + metrics**
- Integrate lifecycle pass into consolidation.
- Emit `state/lifecycle-metrics.json`.
- Keep retrieval behavior unchanged.

4. **PR 20D — Retrieval weighting/filtering + docs rollout**
- Apply lifecycle boost/penalty and optional stale filtering.
- Add explain/debug outputs and rollout guide.
- Run hardening gate before merge.

---

## Architecture Changes

### 1) New lifecycle dimensions

Retain existing `status` (`active/superseded/archived`) and add policy-level dimensions:

- `lifecycleState`: `candidate | validated | active | stale | archived`
- `verificationState`: `unverified | user_confirmed | system_inferred | disputed`
- `policyClass`: `ephemeral | durable | protected`
- `lastValidatedAt?: string`
- `decayScore?: number` (0-1)
- `heatScore?: number` (0-1)

Notes:
- `status` remains storage/index state.
- `lifecycleState` is retrieval/promotion semantics.
- Mapping rule: if `status=archived`, lifecycle must be `archived`.

### 2) Lifecycle scoring engine

Add `src/lifecycle.ts`:

- `computeHeat(memory, now, signals)`
- `computeDecay(memory, now, signals)`
- `decideLifecycleTransition(memory, policy, now)`

Signals consumed (already available in Engram):
- recency (`created`, `updated`, `lastAccessed`)
- usage (`accessCount`)
- confidence (`confidence`, `confidenceTier`)
- feedback (`relevance`, negative examples)
- contradiction/supersession markers
- importance score

### 3) Consolidation integration

In `orchestrator.ts` consolidation flow:

1. load active memories
2. run lifecycle score pass
3. apply transitions (`candidate→validated→active`, `active→stale`, `stale→archived`)
4. persist changed frontmatter (only changed records)
5. emit lifecycle metrics snapshot (`memory/state/lifecycle-metrics.json`)

### 4) Retrieval integration

In retrieval post-processing (`boostSearchResults` + pre-filter path):

- optional filter of `stale` memories (`lifecycleFilterStaleEnabled`)
- additive lifecycle boost/penalty:
  - `validated/active` small boost
  - `candidate` neutral/slight penalty
  - `stale` penalty (or filtered)
  - `disputed` stronger penalty

Fail-open: if lifecycle metadata missing, default to current behavior.

### 5) Config additions

Add to `PluginConfig`, `config.ts`, and `openclaw.plugin.json`:

- `lifecyclePolicyEnabled` (default `false`)
- `lifecycleFilterStaleEnabled` (default `false`)
- `lifecyclePromoteHeatThreshold` (default `0.55`)
- `lifecycleStaleDecayThreshold` (default `0.65`)
- `lifecycleArchiveDecayThreshold` (default `0.85`)
- `lifecycleProtectedCategories` (default: `decision,principle,commitment,preference`)
- `lifecycleMetricsEnabled` (default `true` when policy enabled)

---

## Phased Implementation Steps

### Phase 1 — Data model + parsing/writing (safe foundation)

Files:
- `src/types.ts`
- `src/storage.ts`
- `tests/storage.test.ts`

Tasks:
1. Extend `MemoryFrontmatter` with lifecycle fields (all optional).
2. Update `serializeFrontmatter()` and `parseFrontmatter()`.
3. Ensure unknown/absent lifecycle fields preserve backward compatibility.
4. Add tests for round-trip parsing/serialization.

### Phase 2 — Lifecycle engine module

Files:
- `src/lifecycle.ts` (new)
- `tests/lifecycle.test.ts` (new)

Tasks:
1. Implement deterministic heat/decay functions with bounded outputs.
2. Implement transition decision function with guardrails:
   - never auto-archive protected categories
   - never promote disputed memory to active
   - archived remains terminal unless manual restore
3. Unit tests for edge cases and monotonicity.

### Phase 3 — Consolidation wiring

Files:
- `src/orchestrator.ts`
- `src/config.ts`
- `openclaw.plugin.json`
- `tests/orchestrator.test.ts`

Tasks:
1. Run lifecycle pass near existing archival/cleanup stages.
2. Write only changed frontmatter to minimize churn.
3. Emit `state/lifecycle-metrics.json` with:
   - counts by lifecycle state
   - transition counts per run
   - stale ratio
   - disputed ratio
4. Integration test: sample corpus transitions predictably after simulated time/usage updates.

### Phase 4 — Retrieval policy integration

Files:
- `src/orchestrator.ts` (boost/filter path)
- `tests/retrieval.test.ts`

Tasks:
1. Apply lifecycle penalties/boosts in score post-processing.
2. Optional stale filtering by flag.
3. Ensure no regressions when feature disabled.
4. Add test proving fail-open behavior on legacy memories.

### Phase 5 — Docs + rollout guidance

Files:
- `docs/architecture/memory-lifecycle.md`
- `docs/config-reference.md`
- `docs/setup-config-tuning.md`

Tasks:
1. Document lifecycle state model and config knobs.
2. Add staged rollout recipe:
   - week 1 shadow mode (metrics only)
   - week 2 ranking-only
   - week 3 optional stale filtering

---

## Migration / Compatibility Notes

- Backward compatible: all lifecycle fields optional.
- Legacy memories load as current behavior; inferred defaults only in-memory unless changed.
- No forced reindex required; optional `qmd update` recommended after large transition batches.
- Existing `status` semantics are preserved.
- If `lifecyclePolicyEnabled=false`, behavior remains unchanged.

---

## Risks and Mitigations

1. **Over-demotion of useful memories**
   - Mitigation: start with ranking penalties before hard filtering; protected categories list.

2. **Frontmatter churn causing disk overhead**
   - Mitigation: write only when transition/score delta exceeds threshold.

3. **Policy brittleness across workloads**
   - Mitigation: explicit config thresholds + metrics-driven tuning.

4. **User surprise from hidden retrieval changes**
   - Mitigation: add debug visibility in metrics and optional lifecycle explain output.

5. **Interaction with existing archival logic**
   - Mitigation: lifecycle pass runs before archival; archival remains final gate.

---

## Success Metrics

Measured over 7-day shadow + rollout window:

1. `stale_hit_rate` reduced by >= 20% vs baseline.
2. `recalled_disputed_rate` reduced by >= 40%.
3. Top-5 retrieval precision on internal prompts improves by >= 10%.
4. P95 recall latency increase <= 10%.
5. No increase in failed consolidation passes.

---

## Test / Validation Checklist

- [x] Frontmatter round-trip tests include new lifecycle fields.
- [x] Lifecycle scoring bounded in [0,1], deterministic for fixed inputs.
- [x] Transition rules respect protected categories.
- [x] Disabled flag path is byte-for-byte equivalent in retrieval ordering baseline tests.
- [x] Consolidation integration writes lifecycle metrics file correctly.
- [x] Retrieval tests show stale/disputed penalties applied as configured.
- [x] Fail-open behavior: malformed lifecycle fields do not break extraction/recall.
- [x] Preflight passes (`npm run preflight:quick`).

---

## Dependency Positioning

This plan is intended to execute **after current v8.2 plans**. It is independent of graph internals but complements PR17/PR18/PR19 by adding policy governance on top of richer retrieval structures.
