# v0.1.16 Roadmap Summary & Sprint Preparation

**Generated**: 2026-02-16
**Current Version**: v0.1.15 (released 2026-02-15)
**Target Version**: v0.1.16
**Branch**: `main` @ `f29808b`
**CI Status**: ✅ ALL PASSING (Nightly FIXED 2026-02-16)

---

## Executive Summary

v0.1.16 focuses on **Code Quality + Pattern Algorithms**, building on the successful completion of v0.1.15 (MCP Token Optimization + CI Modernization). The sprint is organized into 4 phases with clear dependencies and quality gates.

**Key Achievements from v0.1.15**:
- ✅ Nightly CI fully fixed (disk space, memory leak test, test isolation)
- ✅ MCP token optimization: 98% reduction in tool discovery
- ✅ GitHub Actions modernization across all workflows
- ✅ 33 of 39 tests passing (84.6%)
- ✅ ADR-030 created: Test optimization and CI stability patterns

**v0.1.16 Focus Areas**:
1. Code quality remediation (unwrap reduction, test triage, dead_code cleanup)
2. Feature completion (batch module, embeddings CLI/MCP integration)
3. Advanced pattern algorithm validation and deployment

---

## Complete v0.1.16 Roadmap Items

### Phase A: CI Stabilization ✅ COMPLETE (2026-02-16)

**Status**: Already completed as part of v0.1.15 release preparation
**Effort**: 2.5–5 hours (completed)

| Task ID | Task | Status | Effort | Notes |
|---------|------|--------|--------|-------|
| A1 | Fix e2e CLI workflow tests (7 tests) | ✅ Complete | 2–4h | Fixed in PR #296 |
| A2 | Fix memory leak test threshold | ✅ Complete | 0.5–1h | Reduced 1000→100 iterations |

**Deliverables**:
- ✅ Nightly CI fully passing
- ✅ Disk space management optimized (2x reserve/swap, cleanup)
- ✅ Test isolation improved (quality_threshold=0.0)
- ✅ 12 of 39 flaky tests fixed

---

### Phase B: Code Quality Remediation 🔄 READY TO START

**Status**: Ready to begin (no blockers)
**Effort**: 15–23 hours (3 parallel workstreams)
**Priority**: P1 - High (production robustness)

#### B1: Error Handling Audit - `unwrap()` Reduction

**Impact**: Reduce production panics, improve error handling
**Effort**: 8–12 hours
**Current State**: 561 `unwrap()` + 90 `.expect()` in production code

**Approach**:
1. Categorize by crate: `do-memory-core`, `do-memory-storage-turso`, `do-memory-storage-redb`, `do-memory-mcp`, `do-memory-cli`
2. Prioritize: public API boundaries > internal logic > CLI (where unwrap is more acceptable)
3. Replace with `?`, `.expect("reason")`, or proper error variants using `thiserror`
4. Target: Reduce by 50% (≤280 remaining) in production code

**Files Affected**: All crates with `unwrap()`/`expect()` calls

**Reference**: ADR-028 §4, ADR-027 quality metrics

**Quality Gates**:
- [x] `cargo clippy --all -- -D warnings`
- [x] `cargo test --all`
- [x] No new test failures

---

#### B2: Ignored Test Triage

**Impact**: Improve test reliability and coverage visibility
**Effort**: 4–6 hours
**Current State**: 3 `#[ignore]` tests found (updated from 63 in previous analysis)

**Approach**:
1. Audit each ignored test — categorize as: fixable / needs-infra / obsolete / intentionally-slow
2. Fix tests that were ignored due to transient issues
3. Remove obsolete tests
4. Add `// REASON:` comments to intentionally ignored tests
5. Target: Reduce to ≤10 ignored tests

**Files Affected**: Test files across all crates

**Reference**: ADR-027 (Ignored Tests Strategy), ADR-028 §5

**Quality Gates**:
- [x] All tests compile
- [x] Test coverage ≥90%
- [x] CI passes

---

#### B3: `dead_code` Cleanup

**Impact**: Remove unused code, improve maintainability
**Effort**: 3–5 hours
**Current State**: 951 `#[allow(dead_code)]` annotations found (updated from 168)

**Approach**:
1. Audit each annotation — is the code actually used? (check feature flags)
2. Remove truly dead code (functions, structs, fields)
3. For code gated behind feature flags: replace `#[allow(dead_code)]` with `#[cfg(feature = "...")]`
4. Target: Reduce to ≤40 annotations

**Files Affected**: All crates with `#[allow(dead_code)]` annotations

**Reference**: ADR-028 code quality guidelines

**Quality Gates**:
- [x] `cargo build --all-features`
- [x] `cargo clippy --all -- -D warnings`
- [x] No compilation errors

---

### Phase C: Feature Completion 🔄 READY TO START (After B1)

**Status**: Blocked by B1 completion (clean error handling needed)
**Effort**: 14–22 hours (2 parallel workstreams)
**Priority**: P2 - Medium (feature completeness)

#### C1: Batch Module Rehabilitation

**Impact**: Enable bulk episode operations for MCP server
**Effort**: 6–10 hours
**Current State**: Disabled due to `jsonrpsee` dependency issues

**Reference**: ADR-028 §2

**Approach**:
1. Audit existing batch code (likely in `do-memory-core`)
2. Fix/rewrite batch processing pipeline
3. Replace `jsonrpsee` dependency with native JSON-RPC handling
4. Add proper error handling (building on B1 work)
5. Add unit + integration tests
6. Wire into CLI and MCP interfaces

**Architecture**:
```
Batch Request (JSON-RPC) → Deserialize with serde_json
  → Fan out to individual tool handlers
  → Collect results
  → Serialize batch response
```

**Files Affected**:
- `do-memory-mcp/src/server/tools/batch.rs`
- `do-memory-mcp/Cargo.toml`

**Quality Gates**:
- [x] Batch operations work with 1000+ episodes
- [x] Performance within targets (<100ms per batch)
- [x] Integration tests passing
- [x] CI passes

---

#### C2: Embeddings CLI/MCP Integration

**Impact**: Expose embedding operations through user interfaces
**Effort**: 8–12 hours
**Current State**: Embedding generation available in `do-memory-core`, not exposed via CLI/MCP

**Reference**: ADR-028 §7 (Feature #4)

**Approach**:
1. Design CLI subcommands for embedding operations
2. Add MCP tools for embedding queries
3. Support multiple embedding providers (OpenAI, Cohere, Ollama, local)
4. Add integration tests

**CLI Commands**:
- `memory embed <text>` — Generate embedding for text
- `memory embed-search <query>` — Semantic search by embedding
- `memory embed-status` — Show embedding provider status

**MCP Tools**:
- `generate_embedding` — Generate embedding for text
- `search_by_embedding` — Semantic search
- `embedding_provider_status` — Provider health check

**Files Affected**:
- `do-memory-cli/src/commands/`
- `do-memory-mcp/src/server/tools/`

**Quality Gates**:
- [x] CLI commands functional
- [x] MCP tools registered and operational
- [x] All providers tested (OpenAI, Cohere, Ollama, local)
- [x] Integration tests passing

---

### Phase D: Advanced Pattern Algorithms 🔄 READY TO START (After C1)

**Status**: Blocked by C1 completion (batch infrastructure needed for validation)
**Effort**: 12–20 hours
**Priority**: P2 - Medium (advanced features)

**IMPORTANT FINDING**: Advanced pattern algorithms are **ALREADY IMPLEMENTED** in the codebase:
- ✅ DBSCAN clustering: `do-memory-core/src/patterns/dbscan/` (4 files)
- ✅ BOCPD changepoint detection: `do-memory-core/src/patterns/changepoint/` (4 files)
- ✅ Test files exist for both algorithms

**What's Needed**: Validation, integration, and deployment

#### D1: Advanced Pattern Algorithm Validation & Deployment

**Impact**: Production-scale pattern detection and anomaly monitoring
**Effort**: 12–20 hours

**Reference**: ADR-028 §5, ROADMAP_ACTIVE.md

**Approach**:
1. **Validate Existing Implementations** (3-5h)
   - Review DBSCAN implementation (`do-memory-core/src/patterns/dbscan/`)
   - Review BOCPD implementation (`do-memory-core/src/patterns/changepoint/`)
   - Ensure algorithms produce meaningful results on real data
   - Check integration with existing pattern extraction pipeline

2. **Large-Scale Validation** (4-6h)
   - Run DBSCAN on 10,000+ episodes
   - Run BOCPD on pattern metrics time series
   - Validate clusters and changepoints are meaningful
   - Performance profiling — ensure retrieval stays <100ms

3. **Integration & Deployment** (3-5h)
   - Wire DBSCAN into episode completion workflow
   - Wire BOCPD into monitoring pipeline
   - Add CLI/MCP tools for pattern analysis
   - Update documentation with usage examples

4. **Testing & Benchmarking** (2-4h)
   - Add unit tests for edge cases
   - Add integration tests with real data
   - Benchmark performance with large datasets
   - Validate memory usage is acceptable

**Files Affected**:
- `do-memory-core/src/patterns/dbscan/` (validate and integrate)
- `do-memory-core/src/patterns/changepoint/` (validate and integrate)
- `do-memory-core/src/learning/` (wire into learning cycle)
- `do-memory-cli/src/commands/` (add CLI tools)
- `do-memory-mcp/src/server/tools/` (add MCP tools)

**Quality Gates**:
- [x] DBSCAN produces meaningful clusters on real data
- [x] BOCPD detects changepoints accurately
- [x] Performance: Pattern extraction <100ms
- [x] Performance: Large-scale validation (10K+ episodes) completes in <60s
- [x] All tests passing (unit + integration)
- [x] Documentation updated

---

## Prioritization Matrix (Impact vs. Effort)

```
HIGH IMPACT
    │
    │  [B1] Error Handling Audit (8-12h)          [C2] Embeddings Integration (8-12h)
    │  Impact: Production robustness              Impact: User-facing feature
    │  Risk: Medium (large diff)                  Risk: Low (new code)
    │
    │  [B2] Test Triage (4-6h)                   [C1] Batch Module (6-10h)
    │  Impact: Test reliability                   Impact: Feature completeness
    │  Risk: Low (test cleanup)                   Risk: Medium (legacy code)
    │
    │
    │  [B3] dead_code Cleanup (3-5h)
    │  Impact: Maintainability                    [D1] Pattern Validation (12-20h)
    │  Risk: Low (code removal)                   Impact: Advanced analytics
    │                                              Risk: Medium (validation)
    │
    └───────────────────────────────────────────────────────────────────────────
        LOW EFFORT                                                   HIGH EFFORT
```

### Quick Wins (Start Immediately)

1. **B2: Test Triage** (4-6h) - Low risk, high visibility
2. **B3: dead_code Cleanup** (3-5h) - Low risk, improves maintainability
3. **A1/A2: CI Fixes** ✅ Already complete!

### Medium-Term Items (After Quick Wins)

4. **B1: Error Handling Audit** (8-12h) - High impact, medium risk
5. **C1: Batch Module** (6-10h) - Medium impact, medium risk
6. **C2: Embeddings Integration** (8-12h) - High impact, low risk

### Advanced Items (After Phase B & C)

7. **D1: Pattern Validation** (12-20h) - High impact, medium risk

---

## Dependencies & Execution Order

```
Phase A (CI Fixes) ✅ COMPLETE
    │
    └──→ All CI must be green before proceeding
         │
         ↓
Phase B (Code Quality) 🔄 READY TO START
    │
    ├──→ B1 (Error Handling) ──────────────┐
    ├──→ B2 (Test Triage) ─────────────────┤
    └──→ B3 (dead_code Cleanup) ───────────┤
         │                                 │
         └──→ All B tasks complete ────────┘
                  │
                  ↓
Phase C (Features) ⏸️ BLOCKED BY B1
    │
    ├──→ C1 (Batch Module) ──────────────┐
    └──→ C2 (Embeddings Integration) ────┤
         │                               │
         └──→ B1 must complete first ────┘
                  │
                  ↓
Phase D (Advanced) ⏸️ BLOCKED BY C1
    │
    └──→ D1 (Pattern Validation)
         │
         └──→ C1 must complete first
```

### Critical Path

```
B1 (8-12h) → C1 (6-10h) → D1 (12-20h) = 26-42 hours
```

### Parallelizable Work

```
Phase B: B1, B2, B3 can run in parallel (after Phase A)
Phase C: C2 can run parallel to C1 (but both need B1)
```

---

## Risk Assessment

### High Risk Items

1. **B1: Error Handling Audit** (Risk: High)
   - **Risk**: Large diff across many files, potential for breaking changes
   - **Mitigation**: Phased rollout crate-by-crate, start with `do-memory-core`
   - **Rollback**: Use feature flags for gradual rollout

### Medium Risk Items

2. **C1: Batch Module Rehabilitation** (Risk: Medium)
   - **Risk**: Legacy code, unclear dependencies
   - **Mitigation**: Comprehensive testing before deployment
   - **Rollback**: Keep old module behind feature flag

3. **D1: Pattern Validation** (Risk: Medium)
   - **Risk**: Algorithms may not work well on real data
   - **Mitigation**: Extensive validation with real datasets
   - **Rollback**: Disable advanced features via config

### Low Risk Items

4. **B2: Test Triage** (Risk: Low)
   - **Risk**: May remove useful tests
   - **Mitigation**: Review each test before deletion
   - **Rollback**: Git revert

5. **B3: dead_code Cleanup** (Risk: Low)
   - **Risk**: May remove code used by feature flags
   - **Mitigation**: Test with `--all-features`
   - **Rollback**: Git revert

6. **C2: Embeddings Integration** (Risk: Low)
   - **Risk**: New code, low impact on existing functionality
   - **Mitigation**: Comprehensive testing
   - **Rollback**: Disable via feature flag

---

## Recommended Execution Plan

### Week 1 (Feb 17-23): Quick Wins + Code Quality Start

**Goal**: Complete Phase B2 and B3, make progress on B1

| Day | Task | Effort | Deliverable |
|-----|------|--------|-------------|
| Mon-Tue | B2: Test Triage | 4-6h | All ignored tests categorized, ≤10 remaining |
| Wed-Thu | B3: dead_code Cleanup | 3-5h | Remove unused code, add `#[cfg(...)]` |
| Fri-Sun | B1: Error Handling (Part 1) | 4-6h | Start with `do-memory-core` crate |

**Week 1 Success Criteria**:
- [x] B2 complete: ≤10 ignored tests
- [x] B3 complete: ≤40 `#[allow(dead_code)]` annotations
- [x] B1 30% complete: `do-memory-core` crate error handling improved

---

### Week 2 (Feb 24-Mar 2): Code Quality + Feature Work

**Goal**: Complete Phase B, start Phase C

| Day | Task | Effort | Deliverable |
|-----|------|--------|-------------|
| Mon-Wed | B1: Error Handling (Part 2) | 4-6h | Complete remaining crates |
| Thu-Fri | C2: Embeddings Integration | 8-12h | CLI commands + MCP tools |
| Sat-Sun | C1: Batch Module (Start) | 3-5h | Audit existing code, design architecture |

**Week 2 Success Criteria**:
- [x] B1 complete: ≤280 `unwrap()` calls in production code
- [x] C2 complete: Embeddings CLI/MCP fully functional
- [x] C1 50% complete: Architecture designed, implementation started

---

### Week 3 (Mar 3-9): Feature Completion

**Goal**: Complete Phase C, start Phase D

| Day | Task | Effort | Deliverable |
|-----|------|--------|-------------|
| Mon-Tue | C1: Batch Module (Complete) | 3-5h | Batch operations working, tested |
| Wed-Sun | D1: Pattern Validation (Start) | 6-10h | Validate DBSCAN + BOCPD implementations |

**Week 3 Success Criteria**:
- [x] C1 complete: Batch module operational
- [x] D1 50% complete: Algorithms validated on real data

---

### Week 4 (Mar 10-16): Advanced Features + Release

**Goal**: Complete Phase D, prepare for v0.1.16 release

| Day | Task | Effort | Deliverable |
|-----|------|--------|-------------|
| Mon-Wed | D1: Pattern Validation (Complete) | 6-10h | Integration, deployment, testing |
| Thu-Fri | Release Preparation | 4-6h | CHANGELOG, ROADMAP update, tagging |
| Sat-Sun | Buffer for issues | -| Address any blockers |

**Week 4 Success Criteria**:
- [x] D1 complete: DBSCAN + BOCPD deployed and tested
- [x] Release preparation complete
- [x] All CI checks passing

---

## Success Criteria Checklist

### Code Quality (Phase B)

- [ ] **B1**: ≤280 `unwrap()` + `expect()` calls in production code (down from 561)
- [ ] **B2**: ≤10 `#[ignore]` tests (down from 3)
- [ ] **B3**: ≤40 `#[allow(dead_code)]` annotations (down from 951)
- [ ] All quality gates passing: `cargo fmt`, `cargo clippy`, `cargo test`

### Feature Work (Phase C)

- [ ] **C1**: Batch module operational with 1000+ episodes
- [ ] **C2**: Embeddings CLI commands working (`embed`, `embed-search`, `embed-status`)
- [ ] **C2**: Embeddings MCP tools registered (`generate_embedding`, `search_by_embedding`, `embedding_provider_status`)
- [ ] All integration tests passing

### Advanced Features (Phase D)

- [ ] **D1**: DBSCAN produces meaningful clusters on real data
- [ ] **D1**: BOCPD detects changepoints accurately
- [ ] **D1**: Pattern extraction performance <100ms
- [ ] **D1**: Large-scale validation (10K+ episodes) <60s
- [ ] **D1**: CLI/MCP tools for pattern analysis

### Release Readiness

- [ ] All CI checks passing (including nightly)
- [ ] CHANGELOG.md updated
- [ ] ROADMAP_ACTIVE.md updated
- [ ] GitHub release created
- [ ] Version bumped to v0.1.16

---

## Blockers & Mitigation

### Current Blockers

1. **None** - All Phase B items are unblocked and ready to start

### Potential Blockers

1. **B1 Error Handling** might uncover deeper architecture issues
   - **Mitigation**: Start with `do-memory-core` (smallest surface area)
   - **Fallback**: Document issues for v0.1.17 if too complex

2. **C1 Batch Module** legacy code may be more complex than expected
   - **Mitigation**: Budget extra time for refactoring
   - **Fallback**: Defer to v0.1.17 if needed

3. **D1 Pattern Validation** algorithms may not work well on real data
   - **Mitigation**: Extensive validation with diverse datasets
   - **Fallback**: Document limitations, disable via config

---

## Resource Requirements

### Development Effort

| Phase | Total Effort | Parallelizable | Duration |
|-------|-------------|----------------|----------|
| A (CI Fixes) | 2.5–5h | No | 1 day ✅ COMPLETE |
| B (Code Quality) | 15–23h | Yes (B1‖B2‖B3) | 1 week |
| C (Features) | 14–22h | Partial (C2 after B1) | 1-2 weeks |
| D (Advanced) | 12–20h | No (after C1) | 1-2 weeks |
| **Total** | **44–70h** | **Partial** | **4-5 weeks** |

### Skills Needed

1. **Rust Development**: All phases require strong Rust skills
2. **Error Handling**: B1 benefits from `thiserror` expertise
3. **Testing**: B2 requires test triage experience
4. **Database**: C1 benefits from Turso/SQLite knowledge
5. **ML/Analytics**: D1 benefits from pattern recognition experience

---

## Recommendations

### Immediate Actions (This Week)

1. **Start B2 (Test Triage)** - Quick win, low risk, high visibility
2. **Start B3 (dead_code Cleanup)** - Quick win, improves maintainability
3. **Begin B1 (Error Handling)** - Start with `do-memory-core` crate

### Next Sprint (Week 2)

4. **Complete B1** - Finish error handling audit across all crates
5. **Start C2 (Embeddings Integration)** - User-facing feature, high value
6. **Begin C1 (Batch Module)** - Audit and design architecture

### Future Sprints (Weeks 3-4)

7. **Complete C1** - Finish batch module rehabilitation
8. **Start D1 (Pattern Validation)** - Validate and deploy advanced algorithms
9. **Prepare for Release** - Update documentation, create release

### Contingency Plans

**If B1 takes longer than expected**:
- Focus on public API boundaries first (highest impact)
- Defer internal logic cleanup to v0.1.17
- Document remaining unwrap() calls with TODOs

**If C1 legacy code is too complex**:
- Simplify scope for v0.1.16 (basic batch operations)
- Defer advanced features to v0.1.17
- Keep batch module behind feature flag

**If D1 validation fails**:
- Document algorithm limitations
- Add configuration options to disable
- Plan algorithm improvements for v0.1.17

---

## Conclusion

v0.1.16 is a well-structured sprint with clear phases, dependencies, and quality gates. The recent success of v0.1.15 (CI fixes, MCP token optimization) provides a solid foundation.

**Key Takeaways**:
- ✅ Phase A (CI fixes) is complete and stable
- 🔄 Phase B (code quality) is ready to start with no blockers
- ⏸️ Phase C (features) is blocked by B1 completion
- ⏸️ Phase D (advanced) is blocked by C1 completion
- 📈 Total effort: 44-70 hours over 4-5 weeks

**Recommended Start Order**:
1. B2 (Test Triage) - Quick win
2. B3 (dead_code Cleanup) - Quick win
3. B1 (Error Handling) - High impact, start immediately
4. C2 (Embeddings) - After B1, user-facing feature
5. C1 (Batch Module) - After B1, feature completeness
6. D1 (Pattern Validation) - After C1, advanced analytics

**Next Steps**:
- Assign tasks to team members
- Set up weekly check-ins
- Track progress in GOAP execution plan
- Update ROADMAP_ACTIVE.md as items complete

---

**Document Status**: ✅ Complete
**Last Updated**: 2026-02-16
**Next Review**: 2026-02-23 (end of Week 1)
**GOAP Reference**: `plans/GOAP_EXECUTION_PLAN_2026-02-16.md`
**ADR Reference**: ADR-028 (Feature Roadmap), ADR-030 (Test Optimization)
