# v3.4.0 roadmap — path to 90+ on LoCoMo

## Current state (2026-04-21)

| | Score | Notes |
|---|---|---|
| v1.1.0 | 70.54 | Original baseline |
| v3.2.1 (conv-0) | 76.7 | BM25 + LLM answerer |
| v3.3.0 (conv-0) | 77.06 | BM25 only (shipped features unused in bench) |
| v3.3.0 features + bench | 70.05 | **Regressed** — decomp+rerank+self-consistency hurt multi-hop |
| SOTA (Mem0/Letta/Zep) | 66-74 | We're already above |
| Target v3.4.0 | 85-90 | Cross-model consensus lift below |

## Cross-model consensus (recent research)

### Tier 1 — 4★+ consensus, ship first

#### 1. Hybrid BM25 + BGE-large-en-v1.5 with RRF (★★★★★ 4.75)
Semantic recall complements BM25 lexical matching. RRF k=60 fusion.  
**Why it works:** LoCoMo queries use paraphrased language; BGE catches semantic matches BM25 misses. Unanimous top pick.  
**Implementation:** Already have `hybrid_recall.HybridBackend`; wire BGE embedding server on `:8090` (requires stopping Ollama temporarily to free RTX 3080 VRAM, OR use BGE-small 512d which fits alongside).

#### 2. LLM query decomposition with UNION retrieval (★★★★ 4.25)
Replace regex `NLPQueryDecomposer` with LLM few-shot decomposition. **KEY:** union the original-question retrieval with sub-query retrievals into a single top-(k×(n_subq+1)) pool. Do NOT RRF-fuse — that's what caused v3.3.0 multi-hop regression.  
**Why it works:** regex can't handle implicit multi-hop; union preserves the bridge-context the fusion dropped.  
**Implementation:** Update `query_planner.LLMQueryDecomposer` to always include `question` as sub_query[0]. Replace RRF-fuse in `hybrid_recall.search` when decomposition active with union+dedup.

#### 3. Iterative retrieval / Chain-of-Retrieval (NEW — a majority of reviewers raised it unprompted)
After initial top-k retrieval, have the answerer generate 1-2 follow-up queries based on what it found and what's missing. Re-retrieve with those. Up to 2 rounds.  
**Why it works:** multi-hop "bridge" questions need second-hop triggered by first-hop evidence. Pre-decomposition guesses; iterative discovers.  
**Implementation:** new module `src/mind_mem/iterative_recall.py` wrapping `hybrid_recall.search`. 2-round cap to bound latency.

### Tier 2 — 3-4★ consensus, ship after Tier 1

#### 4. Graph expansion ±1 hop with decay=0.5 (★★★ 3.75)
Already shipped as `graph_recall.graph_expand` but NOT wired into locomo_judge. Wire it in.  
**Why it works:** LoCoMo has ref-style links; graph expand surfaces blocks missed by BM25+vector.

#### 5. Chain-of-note evidence packing (★★★ 3.75)
Have the answerer summarize retrieved blocks into 3-5 bullet notes before answering.  
**Why it works:** reduces "lost in the middle" on 10-20 retrieved blocks. A reviewer named this specifically.

#### 6. Temporal metadata injection (NEW — raised in review)
Prepend each retrieved block with `[Stored T days ago]` relative to query time. Inject reference date in prompt.  
**Why it works:** our temporal category drops to 39 on full-conv because model has no anchor. This gives it one.

#### 7. Per-category prompts (★★★ 3.75)
Already shipped as `answer_quality.prompt_for_category`. NOT wired into locomo_judge. Wire it in.

### Tier 3 — exploratory, defer

- **LLM reranker** (★★ 2.5) — marginal if BGE already good
- **Paraphrase query expansion** (★★ 2.5) — redundant with hybrid
- **Temp-sampled self-consistency** (★★ 2.25) — a deterministic answerer breaks this
- **Fine-tune encoder on LoCoMo** (★★★ 3.5) — high effort, uncertain lift, skip for v3.4.0

## What v3.3.0 feature experiment taught us

Do NOT ship these without further work:
1. **Regex-only decomposition** (`NLPQueryDecomposer`) — too blunt, misses implicit multi-hop structure.
2. **RRF-fuse of sub-query results** — attenuates the seed question's best hits.
3. **Self-consistency with a temp=0 answerer** — deterministic outputs, 3x cost for zero lift.

These modules stay in the codebase as building blocks (the refactored interfaces are correct), but locomo_judge should NOT invoke them as-is.

## Expected v3.4.0 score trajectory

| Feature added | conv-0 cumulative expected | Confidence |
|---|---|---|
| Current baseline | 77.06 | measured |
| + Hybrid BM25+BGE | ~82 | high (most LLMs 5★) |
| + Union LLM decomposition | ~86 | high (only if UNION not RRF) |
| + Iterative retrieval (2 rounds) | ~88-90 | medium (cost + latency tradeoff) |
| + Graph expansion | ~89-91 | medium |
| + Chain-of-note + per-cat prompts + temporal metadata | ~91-93 | low-medium |

**Realistic LoCoMo ceiling ~95** (5% ambiguous/noisy questions even gold-annotators disagree on).

## Sequencing

1. **v3.3.0 release** (today): bump version, CHANGELOG, tag, PyPI — ship what's already coded.
2. **v3.4.0-alpha (next)**: Tier 1 items (hybrid + union decomp + iterative).
3. **v3.4.0-beta**: Tier 2 items (graph wire-in + chain-of-note + temporal meta + per-cat prompts).
4. **v3.4.0-rc**: full 10-conv sweep, target avg ≥85.
5. **v3.4.0 release**: if rc hits 85+, ship.
