# v8.14 Hot/Cold Parity + Value-Aware Auto-Demotion Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Deliver a fully operational hot/cold memory system where low-value memories are auto-demoted from hot to cold while cold supports the same recall quality and feature set as hot.

**Architecture:** Introduce a tier-routing layer that computes memory value and lifecycle signals, then migrates memories between hot/cold collections with full metadata parity. Use a shared retrieval pipeline so hot and cold candidates receive identical filtering, reranking, budget enforcement, graph enrichment, and telemetry accounting.

**Tech Stack:** TypeScript, QMD collections, orchestrator/retrieval pipeline, lifecycle policy engine, node:test.

---

### Task 1: Define tier-parity contract and config surface

**Files:**
- Modify: `src/types.ts`
- Modify: `src/config.ts`
- Modify: `openclaw.plugin.json`
- Test: `tests/config-cold-qmd.test.ts`
- Docs: `docs/config-reference.md`

**Step 1: Write failing config tests for parity controls**
- Add expectations for new config keys:
  - `qmdTierMigrationEnabled` (default `false`)
  - `qmdTierDemotionMinAgeDays` (default `14`)
  - `qmdTierDemotionValueThreshold` (default `0.35`)
  - `qmdTierPromotionValueThreshold` (default `0.7`)
  - `qmdTierParityGraphEnabled` (default `true`)
  - `qmdTierParityHiMemEnabled` (default `true`)
  - `qmdTierAutoBackfillEnabled` (default `false`)
- Assert explicit `0` values are preserved where applicable.

**Step 2: Run targeted tests to confirm failure**
- Run: `npx tsx --test tests/config-cold-qmd.test.ts`
- Expected: FAIL for missing config fields/defaults.

**Step 3: Implement config + type additions**
- Add parser support and safe defaults in `src/config.ts`.
- Add typed config fields in `src/types.ts`.
- Mirror schema in `openclaw.plugin.json`.

**Step 4: Re-run targeted tests**
- Run: `npx tsx --test tests/config-cold-qmd.test.ts`
- Expected: PASS.

**Step 5: Commit**
- `git add src/types.ts src/config.ts openclaw.plugin.json tests/config-cold-qmd.test.ts docs/config-reference.md`
- `git commit -m "feat: add hot-cold parity and tier migration config contract"`

### Task 2: Implement value scoring for tier routing

**Files:**
- Create: `src/tier-routing.ts`
- Modify: `src/lifecycle.ts`
- Test: `tests/tier-routing.test.ts`
- Docs: `docs/architecture/memory-lifecycle.md`

**Step 1: Write failing tier-routing unit tests**
- Cover value score inputs: confidence, access recency, access count, importance, disputed penalties, correction signals.
- Cover deterministic demote/promote decisions near threshold boundaries.

**Step 2: Run tests to verify failure**
- Run: `npx tsx --test tests/tier-routing.test.ts`
- Expected: FAIL (module/functions missing).

**Step 3: Implement minimal routing evaluator**
- Add `computeTierValueScore()` and `decideTierTransition()` in `src/tier-routing.ts`.
- Reuse lifecycle helpers where possible; avoid duplicate weighting logic.

**Step 4: Re-run tests**
- Run: `npx tsx --test tests/tier-routing.test.ts`
- Expected: PASS.

**Step 5: Commit**
- `git add src/tier-routing.ts src/lifecycle.ts tests/tier-routing.test.ts docs/architecture/memory-lifecycle.md`
- `git commit -m "feat: add value-aware tier routing decisions"`

### Task 3: Build migration executor for hot->cold and cold->hot moves

**Files:**
- Create: `src/tier-migration.ts`
- Modify: `src/storage.ts`
- Modify: `src/qmd.ts`
- Test: `tests/tier-migration.test.ts`

**Step 1: Write failing migration tests**
- Cases:
  - Demotion writes to cold storage path and cold QMD collection.
  - Promotion restores to hot storage path and hot QMD collection.
  - Metadata parity persists (`links`, `verificationState`, `importance`, `lifecycleState`, `threadId`, `memoryKind`).
  - Idempotent re-runs do not duplicate records.

**Step 2: Run tests to verify failure**
- Run: `npx tsx --test tests/tier-migration.test.ts`
- Expected: FAIL.

**Step 3: Implement migration executor**
- Add file move/copy primitives in `src/storage.ts` with atomic write + temp file semantics.
- Add collection-targeted QMD update/embed functions in `src/qmd.ts`.
- Implement migration orchestration in `src/tier-migration.ts` with per-memory journaling.

**Step 4: Re-run tests**
- Run: `npx tsx --test tests/tier-migration.test.ts`
- Expected: PASS.

**Step 5: Commit**
- `git add src/tier-migration.ts src/storage.ts src/qmd.ts tests/tier-migration.test.ts`
- `git commit -m "feat: implement hot-cold migration executor with parity metadata"`

### Task 4: Wire automatic demotion/promotion into extraction + maintenance loops

**Files:**
- Modify: `src/orchestrator.ts`
- Modify: `src/compounding/engine.ts`
- Test: `tests/orchestrator-tier-migration.test.ts`
- Test: `tests/compounding-tier-migration.test.ts`

**Step 1: Write failing orchestration tests**
- Verify migration runs after extraction/maintenance when enabled.
- Verify disabled path is no-op.
- Verify migration is bounded per cycle (no unbounded scan).

**Step 2: Run tests to verify failure**
- Run: `npx tsx --test tests/orchestrator-tier-migration.test.ts tests/compounding-tier-migration.test.ts`
- Expected: FAIL.

**Step 3: Implement orchestration wiring**
- Add migration pass to extraction completion path.
- Add scheduled pass to compounding cycle.
- Add retry/backoff and fail-open behavior.

**Step 4: Re-run tests**
- Run: `npx tsx --test tests/orchestrator-tier-migration.test.ts tests/compounding-tier-migration.test.ts`
- Expected: PASS.

**Step 5: Commit**
- `git add src/orchestrator.ts src/compounding/engine.ts tests/orchestrator-tier-migration.test.ts tests/compounding-tier-migration.test.ts`
- `git commit -m "feat: wire auto tier migration into extraction and maintenance loops"`

### Task 5: Enforce retrieval parity between hot and cold

**Files:**
- Modify: `src/retrieval.ts`
- Modify: `src/rerank.ts`
- Modify: `src/graph.ts`
- Modify: `src/himem.ts`
- Test: `tests/retrieval-hot-cold-parity.test.ts`
- Test: `tests/graph-cold-tier-parity.test.ts`

**Step 1: Write failing retrieval parity tests**
- Ensure hot and cold candidates both flow through:
  - policy filters
  - rerank/boost
  - cap-after-filter behavior
  - graph assist (when enabled)
  - HiMem episode/note treatment
- Ensure artifact isolation remains enforced across both tiers.

**Step 2: Run tests to verify failure**
- Run: `npx tsx --test tests/retrieval-hot-cold-parity.test.ts tests/graph-cold-tier-parity.test.ts`
- Expected: FAIL.

**Step 3: Implement shared candidate pipeline**
- Refactor retrieval to a common candidate normalization function.
- Apply the same parity pipeline to both tiers before final cap.
- Preserve existing planner/recall mode semantics.

**Step 4: Re-run tests**
- Run: `npx tsx --test tests/retrieval-hot-cold-parity.test.ts tests/graph-cold-tier-parity.test.ts`
- Expected: PASS.

**Step 5: Commit**
- `git add src/retrieval.ts src/rerank.ts src/graph.ts src/himem.ts tests/retrieval-hot-cold-parity.test.ts tests/graph-cold-tier-parity.test.ts`
- `git commit -m "feat: enforce retrieval parity for hot and cold tiers"`

### Task 6: Add observability and operator controls

**Files:**
- Modify: `src/cli.ts`
- Modify: `src/recall-state.ts`
- Test: `tests/cli-tier-status.test.ts`
- Docs: `docs/operations.md`

**Step 1: Write failing CLI tests**
- Add tests for:
  - `openclaw engram tier-status`
  - `openclaw engram tier-migrate --dry-run`
  - `openclaw engram tier-migrate --limit N`

**Step 2: Run tests to verify failure**
- Run: `npx tsx --test tests/cli-tier-status.test.ts`
- Expected: FAIL.

**Step 3: Implement CLI + telemetry**
- Add migration counters and per-cycle summaries.
- Emit migration reasons and error counts without leaking sensitive content.

**Step 4: Re-run tests**
- Run: `npx tsx --test tests/cli-tier-status.test.ts`
- Expected: PASS.

**Step 5: Commit**
- `git add src/cli.ts src/recall-state.ts tests/cli-tier-status.test.ts docs/operations.md`
- `git commit -m "feat: add tier migration telemetry and operator commands"`

### Task 7: Hardening gate + release docs

**Files:**
- Modify: `docs/ops/pr-review-hardening-playbook.md`
- Modify: `CHANGELOG.md`
- Modify: `docs/setup-config-tuning.md`

**Step 1: Add hardening checks specific to tier parity**
- Include contract checks for cap-after-filter, artifact isolation, and `0` semantics.

**Step 2: Run full verification**
- Run:
  - `npm run check-types`
  - `npm test`
  - `npm run build`
  - `npm run preflight:quick`
- Expected: all PASS.

**Step 3: Commit**
- `git add docs/ops/pr-review-hardening-playbook.md CHANGELOG.md docs/setup-config-tuning.md`
- `git commit -m "docs: add rollout and hardening guidance for hot-cold parity"`

## Rollout Notes

1. Start with `qmdTierMigrationEnabled=false` in production.
2. Enable in canary with strict migration limits.
3. Observe tier status + recall quality for 48 hours.
4. Then enable auto-backfill and graph parity in cold tier.
