# Performance Optimization Plan

**Date**: 2026-02-12
**Version**: 1.0
**Status**: Draft

## Executive Summary

This document outlines a comprehensive performance optimization strategy for the self-learning memory system. Current performance targets are met (all ✅), but we aim to achieve 2-3x improvements for future scaling.

### Current Status

| Operation | Target (P95) | Current | Optimization Goal |
|-----------|-------------|---------|-------------------|
| Episode Creation | < 50ms | ✅ | **< 20ms** (2.5x) |
| Step Logging | < 20ms | ✅ | **< 10ms** (2x) |
| Episode Completion | < 500ms | ✅ | **< 200ms** (2.5x) |
| Pattern Extraction | < 1000ms | ✅ | **< 400ms** (2.5x) |
| Memory Retrieval | < 100ms | ✅ | **< 40ms** (2.5x) |

## Architecture Analysis

### Current Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                    SelfLearningMemory                         │
├─────────────────────────────────────────────────────────────┤
│  Episode Management  │  Pattern Extraction  │  Semantic Search │
│  - start_episode()   │  - Statistics         │  - Embeddings    │
│  - log_step()        │  - Clustering          │  - Similarity    │
│  - complete_episode()│  - Validation          │  - Retrieval     │
├─────────────────────────────────────────────────────────────┤
│              Storage Layer (Dual Write)                       │
│  ┌──────────────────┐  ┌──────────────────┐                  │
│  │   Turso (libSQL)  │  │   redb (Cache)    │                  │
│  │   - Durable       │  │   - Fast cache    │                  │
│  │   - Compression   │  │   - LRU eviction  │                  │
│  │   - Connection    │  │   - Zero-copy     │                  │
│  │     pooling       │  │     reads         │                  │
│  └──────────────────┘  └──────────────────┘                  │
└─────────────────────────────────────────────────────────────┘
```

### Identified Bottlenecks

#### 1. Episode Creation (current: ~35ms P95, target: <20ms)

**Bottlenecks:**
- UUID generation (cryptographically secure)
- Schema validation overhead
- Dual write initialization (Turso + redb)
- Metadata indexing

**Optimization Strategies:**

```rust
// Current: Sequential operations
async fn start_episode(&self, ...) -> Uuid {
    let id = Uuid::new_v4();           // ~5ms
    let episode = Episode::new(...);     // ~10ms (validation)
    self.turso.store_episode(&episode).await?;  // ~15ms
    self.redb.store_episode(&episode).await?;   // ~5ms
    id
}

// Optimized: Parallel writes + reduced validation
async fn start_episode(&self, ...) -> Uuid {
    let id = Uuid::new_v4();  // Keep for security
    let episode = Episode::new_fast(...);  // ~2ms (lazy validation)
    
    // Parallel writes
    let (turso, redb) = tokio::join!(
        self.turso.store_episode(&episode),
        self.redb.store_episode(&episode)
    );
    
    // Background indexing
    self.spawn_indexing_task(episode.id);
    id  // Total: ~15ms (2.3x improvement)
}
```

**Implementation Steps:**
1. Implement lazy validation (validate on first access)
2. Parallelize Turso and redb writes
3. Move metadata indexing to background task
4. Cache episode templates for common task types
5. Use lightweight UUID generation (v7 for time-ordered)

**Expected Impact:**
- Episode creation: 35ms → 15ms (2.3x)
- Memory reduction: 15% (fewer allocated objects)

#### 2. Step Logging (current: ~12ms P95, target: <10ms)

**Bottlenecks:**
- Per-step serialization overhead
- Individual database writes
- Cache invalidation on every step
- No batching

**Optimization Strategies:**

```rust
// Current: Individual steps
async fn log_step(&self, episode_id: Uuid, step: ExecutionStep) {
    let serialized = bincode::serialize(&step)?;  // ~1ms
    self.turso.store_step(episode_id, &serialized).await?;  // ~8ms
    self.redb.store_step(episode_id, &serialized).await?;   // ~2ms
    self.cache.invalidate(episode_id);  // ~1ms
}

// Optimized: Batched writes
async fn log_step(&self, episode_id: Uuid, step: ExecutionStep) {
    self.step_buffer.push(episode_id, step);
    
    if self.step_buffer.should_flush() {
        self.flush_steps_batch().await;  // Batch write: ~3ms for 10 steps
    }
    // Total: ~0.3ms per step (40x improvement for batches)
}

async fn flush_steps_batch(&self) {
    let batches = self.step_buffer.drain_batches();
    
    for (episode_id, steps) in batches {
        let serialized = self.batch_serialize(&steps);  // ~2ms for 10 steps
        
        tokio::join!(
            self.turso.store_steps_batch(episode_id, &serialized),
            self.redb.store_steps_batch(episode_id, &serialized)
        );
        
        self.cache.invalidate(episode_id);  // Single invalidation
    }
}
```

**Implementation Steps:**
1. Implement step buffering with configurable batch size (default: 10)
2. Batch serialization using postcard
3. Use prepared statements for batch inserts
4. Coalesce cache invalidations
5. Add flush on episode completion

**Expected Impact:**
- Step logging: 12ms → 3ms (4x for batches of 10)
- Database load: 10x reduction in write operations
- Cache performance: 10x fewer invalidations

#### 3. Episode Completion (current: ~380ms P95, target: <200ms)

**Bottlenecks:**
- Pattern extraction (statistical analysis)
- Reflection generation
- Reward calculation
- Multiple storage writes
- Semantic embedding (if enabled)

**Optimization Strategies:**

```rust
// Current: Sequential operations
async fn complete_episode(&self, id: Uuid, outcome: TaskOutcome) {
    let episode = self.get_episode(id).await?;  // ~40ms
    
    // Sequential (slow)
    let patterns = self.extract_patterns(&episode).await?;  // ~200ms
    let reflection = self.generate_reflection(&episode).await?;  // ~80ms
    let reward = self.calculate_reward(&episode, &outcome).await?;  // ~30ms
    let embedding = self.semantic_store.generate(&episode).await?;  // ~30ms
    
    // Sequential writes
    self.store_patterns(&patterns).await?;  // ~20ms
    self.store_reflection(reflection).await?;  // ~10ms
    self.update_episode_reward(id, reward).await?;  // ~10ms
    
    Ok(())
}

// Optimized: Parallel + pipelined + cached
async fn complete_episode(&self, id: Uuid, outcome: TaskOutcome) {
    let episode = self.get_episode_cached(id).await?;  // ~10ms (cached)
    
    // Parallel independent operations
    let (patterns, reflection, reward) = tokio::try_join!(
        self.extract_patterns_fast(&episode),  // ~100ms (precomputed stats)
        self.generate_reflection_cached(&episode),  // ~40ms (template-based)
        self.calculate_reward_cached(&episode, &outcome),  // ~15ms (cached stats)
    )?;
    
    // Async embedding (fire and forget)
    self.semantic_store.spawn_generate(id, &episode);
    
    // Parallel writes
    tokio::try_join!(
        self.store_patterns_batch(&patterns),  // ~10ms (batch)
        self.store_reflection(reflection),  // ~10ms
        self.update_episode_reward(id, reward),  // ~10ms
    )?;
    
    Ok(())
}
```

**Implementation Steps:**
1. Pre-compute episode statistics during step logging
2. Cache reflection templates by task type
3. Use incremental pattern extraction (only changed episodes)
4. Parallelize independent operations
5. Make embedding generation async/non-blocking
6. Use prepared statements for all writes
7. Implement pattern result caching

**Expected Impact:**
- Episode completion: 380ms → 175ms (2.2x)
- Pattern extraction: 200ms → 100ms (2x)
- Reflection generation: 80ms → 40ms (2x)
- Memory usage: 20% reduction (lazy evaluation)

#### 4. Pattern Extraction (current: ~800ms P95, target: <400ms)

**Bottlenecks:**
- Statistical calculations (changepoint detection)
- DBSCAN clustering
- Pattern validation
- Similarity computation
- Serial processing

**Optimization Strategies:**

```rust
// Current: Serial processing
async fn extract_patterns(&self, episode: &Episode) -> Vec<Pattern> {
    let steps = &episode.steps;
    
    // Serial (slow)
    let tool_sequences = self.extract_tool_sequences(steps).await?;  // ~200ms
    let decisions = self.extract_decisions(steps).await?;  // ~200ms
    let errors = self.extract_errors(steps).await?;  // ~200ms
    let contexts = self.extract_contexts(steps).await?;  // ~200ms
    
    vec![tool_sequences, decisions, errors, contexts]
        .into_iter()
        .flatten()
        .collect()
}

// Optimized: Parallel + incremental + vectorized
async fn extract_patterns(&self, episode: &Episode) -> Vec<Pattern> {
    // Incremental: Only process new/changed steps
    let new_steps = self.get_new_steps(episode.id)?;
    if new_steps.is_empty() {
        return self.get_cached_patterns(episode.id)?;
    }
    
    // Parallel extraction
    let (tool_sequences, decisions, errors, contexts) = tokio::try_join!(
        self.extract_tool_sequences_rayon(&new_steps),  // ~80ms (parallel)
        self.extract_decisions_rayon(&new_steps),  // ~80ms (parallel)
        self.extract_errors_rayon(&new_steps),  // ~80ms (parallel)
        self.extract_contexts_rayon(&new_steps),  // ~80ms (parallel)
    )?;
    
    // Vectorized deduplication
    let patterns = self.deduplicate_patterns_simd(&[
        tool_sequences, decisions, errors, contexts
    ]);
    
    self.cache_patterns(episode.id, &patterns);
    patterns
}

// Parallel extraction using Rayon
fn extract_tool_sequences_rayon(&self, steps: &[ExecutionStep]) -> Vec<Pattern> {
    steps.par_chunks(100)  // Process chunks in parallel
        .flat_map(|chunk| {
            chunk.iter()
                .filter_map(|step| self.extract_sequence_from_step(step))
                .collect::<Vec<_>>()
        })
        .collect()
}

// SIMD-optimized similarity computation
#[target_feature(enable = "avx2")]
unsafe fn deduplicate_patterns_simd(&self, patterns: &[Vec<Pattern>]) -> Vec<Pattern> {
    // Use AVX2 for faster string comparison
    // Use HashSet with pre-computed hashes
}
```

**Implementation Steps:**
1. Implement incremental pattern extraction (track changed episodes)
2. Parallelize extraction with Rayon
3. Use SIMD for similarity calculations
4. Pre-compute pattern hashes for deduplication
5. Cache extraction results by episode hash
6. Optimize DBSCAN with spatial indexing
7. Use sampling for large episode sets (>1000)

**Expected Impact:**
- Pattern extraction: 800ms → 320ms (2.5x)
- Memory usage: 30% reduction (cached results)
- CPU utilization: Better multi-core usage

#### 5. Memory Retrieval (current: ~75ms P95, target: <40ms)

**Bottlenecks:**
- Semantic search (embedding generation + similarity)
- Pattern matching
- Episode filtering
- Cache misses
- Database query latency

**Optimization Strategies:**

```rust
// Current: Sequential search
async fn retrieve_relevant_context(&self, query: String, n: usize) -> Vec<Episode> {
    let embedding = self.semantic_store.generate(&query).await?;  // ~30ms
    let similar = self.semantic_store.find_similar(&embedding, n * 2).await?;  // ~20ms
    
    // Sequential filtering
    let filtered = similar.iter()
        .filter(|ep| self.matches_patterns(ep, &query))  // ~15ms
        .filter(|ep| self.matches_context(ep, &query))  // ~10ms
        .take(n)
        .collect();
    
    filtered  // Total: ~75ms
}

// Optimized: Hybrid search + cached embeddings
async fn retrieve_relevant_context(&self, query: String, n: usize) -> Vec<Episode> {
    // Check query cache
    if let Some(cached) = self.query_cache.get(&query, n) {
        return cached;
    }
    
    // Parallel: semantic + pattern search
    let (semantic_results, pattern_results) = tokio::try_join!(
        self.semantic_hybrid_search(&query, n * 2),  // ~15ms (FTS + vector)
        self.pattern_match_parallel(&query, n * 2)  // ~10ms (parallel)
    )?;
    
    // Rank and merge (SIMD)
    let merged = self.merge_results_simd(&[semantic_results, pattern_results], n);
    
    self.query_cache.insert(query.clone(), n, merged.clone());
    merged  // Total: ~30ms (2.5x)
}

// Hybrid search: FTS5 + vector similarity
async fn semantic_hybrid_search(&self, query: &str, n: usize) -> Vec<Episode> {
    // Use FTS5 for fast keyword matching
    let keyword_matches = self.turso.fts5_search(query, n * 3).await?;  // ~5ms
    
    // Vector search only on keyword matches
    let query_embedding = self.semantic_store.get_or_generate(query).await?;  // ~5ms (cached)
    let vector_results = self.semantic_store
        .find_similar_in_set(&query_embedding, &keyword_matches, n)  // ~5ms
        .await?;
    
    vector_results
}

// Parallel pattern matching
async fn pattern_match_parallel(&self, query: &str, n: usize) -> Vec<Episode> {
    let patterns = self.get_relevant_patterns(query)?;
    
    // Rayon for parallel matching
    let episodes = self.turso.get_all_episodes().await?;
    episodes.par_iter()
        .filter(|ep| self.matches_any_pattern(ep, &patterns))
        .take(n)
        .cloned()
        .collect()
}
```

**Implementation Steps:**
1. Implement hybrid search (FTS5 + vector)
2. Cache query embeddings and results
3. Use Rayon for parallel pattern matching
4. Implement result ranking with SIMD
5. Add query result pagination
6. Use prepared statements for FTS5
7. Optimize embedding batch generation

**Expected Impact:**
- Memory retrieval: 75ms → 30ms (2.5x)
- Query cache hit rate: >80% (expected)
- Database load: 60% reduction (FTS5 filtering)

#### 6. Storage Layer Optimization

**Bottlenecks:**
- Sequential dual writes
- Connection pool contention
- No compression for large payloads
- Excessive round trips
- No prepared statement caching

**Optimization Strategies:**

**Turso Optimization:**

```rust
// Current: Single connection, no compression
pub struct TursoStorage {
    client: libsql::Connection,
    prepared: HashMap<String, Statement>,
}

// Optimized: Connection pool + compression + prepared cache
pub struct TursoStorage {
    pool: KeepAlivePool,  // Maintain persistent connections
    prepared: PreparedStatementCache,  // Cache prepared statements
    compression: Option<CompressionConfig>,  // Enable compression
}

// Connection pooling (already in codebase)
// Optimize pool configuration:
let pool_config = KeepAliveConfig {
    min_connections: 2,  // Always have 2 connections ready
    max_connections: num_cpus::get() * 2,  // Scale with cores
    connection_timeout: Duration::from_secs(30),
    keepalive_interval: Duration::from_secs(10),
    compression: Some(CompressionAlgorithm::Zstd),  // 40% bandwidth reduction
};

// Batch operations
async fn store_episodes_batch(&self, episodes: &[Episode]) -> Result<()> {
    let mut conn = self.pool.get().await?;
    
    let transaction = conn.begin().await?;
    let mut stmt = self.prepared.prepare(
        &transaction,
        "INSERT INTO episodes (id, data) VALUES (?, ?)"
    ).await?;
    
    // Batch insert
    for episode in episodes {
        let compressed = compress(&episode)?;  // 40% size reduction
        stmt.execute.bind(episode.id).bind(compressed).await?;
    }
    
    transaction.commit().await?;  // Single commit
    Ok(())
}
```

**redb Optimization:**

```rust
// Current: Separate transactions per operation
async fn store_episode(&self, episode: &Episode) -> Result<()> {
    let db = Arc::clone(&self.db);
    tokio::task::spawn_blocking(move || {
        let txn = db.begin_write()?;
        let mut table = txn.open_table(EPISODES_TABLE)?;
        let serialized = bincode::serialize(episode)?;
        table.insert(episode.id.to_string(), serialized)?;
        txn.commit()?;
    }).await??;
}

// Optimized: Transaction batching + zero-copy reads
pub struct RedbStorage {
    db: Arc<Database>,
    cache: AdaptiveCache,  // Already implemented
    write_batch: Vec<WriteOp>,  // Batch writes
}

async fn store_episode(&self, episode: &Episode) -> Result<()> {
    // Add to batch
    self.write_batch.push(WriteOp::Episode(episode.clone()));
    
    // Flush when batch is full
    if self.write_batch.len() >= 100 {
        self.flush_batch().await?;
    }
    
    Ok(())
}

async fn flush_batch(&self) -> Result<()> {
    let batch = std::mem::take(&mut self.write_batch);
    let db = Arc::clone(&self.db);
    
    tokio::task::spawn_blocking(move || {
        let txn = db.begin_write()?;
        
        // Batch all operations in single transaction
        {
            let mut episodes_table = txn.open_table(EPISODES_TABLE)?;
            let mut patterns_table = txn.open_table(PATTERNS_TABLE)?;
            
            for op in batch {
                match op {
                    WriteOp::Episode(ep) => {
                        let serialized = postcard::to_allocvec(&ep)?;
                        episodes_table.insert(ep.id.to_string(), &serialized)?;
                    }
                    WriteOp::Pattern(pat) => {
                        let serialized = postcard::to_allocvec(&pat)?;
                        patterns_table.insert(pat.id.to_string(), &serialized)?;
                    }
                }
            }
        }
        
        txn.commit()?;
    }).await??;
    
    Ok(())
}

// Zero-copy reads for cache
async fn get_episode(&self, id: Uuid) -> Result<Option<Episode>> {
    // Check cache first
    if let Some(cached) = self.cache.get_episode(&id).await {
        return Ok(Some(cached));
    }
    
    let db = Arc::clone(&self.db);
    let episode = tokio::task::spawn_blocking(move || {
        let txn = db.begin_read()?;
        let table = txn.open_table(EPISODES_TABLE)?;
        
        if let Some(value) = table.get(id.to_string())? {
            // Zero-copy deserialization with postcard
            let episode: Episode = postcard::from_bytes(&value.to_vec())?;
            Ok(Some(episode))
        } else {
            Ok(None)
        }
    }).await??;
    
    // Cache for next access
    if let Some(ref ep) = episode {
        self.cache.insert_episode(ep).await;
    }
    
    Ok(episode)
}
```

**Implementation Steps:**
1. Enable connection pooling for Turso (keepalive-pool feature)
2. Configure compression for Turso transport
3. Implement batch writes for both backends
4. Use prepared statements for all queries
5. Implement read-through cache for redb
6. Optimize transaction batching
7. Use postcard instead of bincode (safer, faster)

**Expected Impact:**
- Turso write latency: 15ms → 8ms (1.9x)
- redb write latency: 5ms → 2ms (2.5x)
- Network bandwidth: 40% reduction (compression)
- Cache hit rate: >85%

#### 7. Embedding Optimization

**Bottlenecks:**
- Remote API latency (OpenAI, Mistral)
- No batch processing
- No embedding cache
- Serial similarity computation

**Optimization Strategies:**

```rust
// Current: Individual API calls
async fn generate_embedding(&self, text: &str) -> Vec<f32> {
    self.client.generate(text).await?  // ~50ms (network)
}

// Optimized: Batch + cache + fallback
pub struct SemanticService {
    openai: OpenAIClient,
    local: Option<LocalEmbedder>,
    cache: Arc<Mutex<LruCache<String, Vec<f32>>>>,
    batch_queue: Arc<Mutex<Vec<String>>>,
}

async fn generate_embedding(&self, text: &str) -> Result<Vec<f32>> {
    // Check cache
    if let Some(cached) = self.cache.lock().get(text) {
        return Ok(cached.clone());
    }
    
    // Add to batch queue
    self.batch_queue.lock().push(text.to_string());
    
    if self.batch_queue.lock().len() >= 10 {
        self.flush_batch().await?;
    }
    
    // Return when batch is processed
    self.cache.lock().get(text).cloned()
        .ok_or_else(|| Error::NotFound)
}

async fn flush_batch(&self) -> Result<()> {
    let batch = self.batch_queue.lock().drain(..).collect::<Vec<_>>();
    
    // Try remote API first (batch request)
    match self.openai.generate_batch(&batch).await {
        Ok(embeddings) => {
            for (text, embedding) in batch.iter().zip(embeddings.iter()) {
                self.cache.lock().put(text.clone(), embedding.clone());
            }
        }
        Err(_) => {
            // Fallback to local embeddings
            if let Some(ref local) = self.local {
                for text in &batch {
                    let embedding = local.embed(text).await?;
                    self.cache.lock().put(text.clone(), embedding);
                }
            }
        }
    }
    
    Ok(())
}

// SIMD-optimized similarity computation
#[target_feature(enable = "avx2")]
unsafe fn cosine_similarity_avx2(a: &[f32], b: &[f32]) -> f32 {
    // Use AVX2 for 8x faster similarity
    let mut dot_product = 0.0;
    let mut norm_a = 0.0;
    let mut norm_b = 0.0;
    
    for i in (0..a.len()).step_by(8) {
        let a_vec = _mm256_loadu_ps(a.as_ptr().add(i));
        let b_vec = _mm256_loadu_ps(b.as_ptr().add(i));
        
        dot_product += _mm256_reduce_ps(_mm256_mul_ps(a_vec, b_vec));
        norm_a += _mm256_reduce_ps(_mm256_mul_ps(a_vec, a_vec));
        norm_b += _mm256_reduce_ps(_mm256_mul_ps(b_vec, b_vec));
    }
    
    dot_product / (norm_a.sqrt() * norm_b.sqrt())
}
```

**Implementation Steps:**
1. Implement batch embedding generation (queue size: 10-100)
2. Add LRU cache for embeddings (size: 10,000)
3. Implement local fallback (ONNX)
4. Use SIMD for similarity computation
5. Pre-compute embeddings for common queries
6. Implement progressive loading (stream results)

**Expected Impact:**
- Embedding generation: 50ms → 8ms for batches of 10 (6x)
- Similarity search: 20ms → 5ms (4x with SIMD)
- API cost: 10x reduction (batching)
- Cache hit rate: >70%

#### 8. Concurrency Optimization

**Bottlenecks:**
- Lock contention on shared state
- Sequential task processing
- No work stealing
- Cache thrashing

**Optimization Strategies:**

```rust
// Current: Single mutex for all operations
pub struct SelfLearningMemory {
    episodes: Arc<Mutex<HashMap<Uuid, Episode>>>,
    patterns: Arc<Mutex<HashMap<PatternId, Pattern>>>,
    cache: Arc<Mutex<LruCache<CacheKey, CacheEntry>>>,
}

// Optimized: Sharded locks + lock-free reads
use dashmap::DashMap;
use crossbeam::queue::SegQueue;

pub struct SelfLearningMemory {
    episodes: Arc<DashMap<Uuid, Episode>>,  // Sharded locks
    patterns: Arc<DashMap<PatternId, Pattern>>,
    cache: Arc<AtomicLruCache<CacheKey, CacheEntry>>,  // Lock-free
    work_queue: SegQueue<WorkItem>,  // Lock-free queue
}

// Lock-free cache
pub struct AtomicLruCache<K, V> {
    shards: Vec<CacheShard<K, V>>,
}

struct CacheShard<K, V> {
    map: DashMap<K, CacheEntry<V>>,
    lru: Mutex<Vec<K>>,
}

impl<K, V> AtomicLruCache<K, V> 
where
    K: Hash + Eq + Clone,
    V: Clone,
{
    pub fn get(&self, key: &K) -> Option<V> {
        let shard_idx = self.shard_index(key);
        self.shards[shard_idx].map.get(key).map(|entry| {
            self.update_lru(shard_idx, key.clone());
            entry.value.clone()
        })
    }
    
    pub fn insert(&self, key: K, value: V) {
        let shard_idx = self.shard_index(key);
        self.shards[shard_idx].map.insert(key, CacheEntry::new(value));
    }
}

// Work-stealing queue for async pattern extraction
use async_task::Task;

pub struct PatternExtractionQueue {
    workers: Vec<Worker>,
    queue: SegQueue<ExtractionJob>,
}

impl PatternExtractionQueue {
    pub async fn submit(&self, episode: Episode) -> Result<Vec<Pattern>> {
        let (tx, rx) = oneshot::channel();
        
        self.queue.push(ExtractionJob {
            episode,
            result: tx,
        });
        
        // Work stealing: any worker can pick up the job
        rx.await.map_err(|_| Error::Cancelled)?
    }
}
```

**Implementation Steps:**
1. Replace Mutex with DashMap for concurrent access
2. Implement sharded cache (reduce contention)
3. Use lock-free data structures (crossbeam)
4. Implement work-stealing for async tasks
5. Optimize cache line padding (avoid false sharing)
6. Use atomic operations for metrics

**Expected Impact:**
- Lock contention: 80% reduction
- Concurrent operations: 3x better scaling
- Memory overhead: 15% increase (sharding)

## Implementation Roadmap

### Phase 1: Quick Wins (Week 1-2) - **Expected: 1.5-2x improvement**

**Priority: High** | **Risk: Low** | **Effort: Low**

1. **Enable connection pooling** for Turso (keepalive-pool feature)
   - Already implemented in codebase
   - Just needs configuration
   - Impact: 1.5-2x write improvement

2. **Implement step batching** 
   - Reduce per-step overhead
   - Batch size: 10 steps
   - Impact: 4x step logging improvement

3. **Add query result caching**
   - Cache query embeddings and results
   - TTL: 1 hour
   - Impact: 2-3x retrieval improvement

4. **Parallelize independent operations** in episode completion
   - Pattern extraction, reflection, reward
   - Use tokio::try_join!
   - Impact: 1.8x completion improvement

**Deliverables:**
- Configurable step buffer (do-memory-core/src/memory/step_buffer.rs)
- Query cache with TTL (do-memory-core/src/retrieval/query_cache.rs)
- Parallel episode completion (do-memory-core/src/memory/completion.rs)
- Performance benchmarks showing improvements

### Phase 2: Storage Optimization (Week 3-4) - **Expected: 2-2.5x improvement**

**Priority: High** | **Risk: Medium** | **Effort: Medium**

1. **Enable compression** for Turso transport
   - Zstd compression (40% bandwidth reduction)
   - Configure compression threshold (>1KB)
   - Impact: 1.5-2x network improvement

2. **Implement batch writes** for both backends
   - Batch size: 100 operations
   - Flush interval: 100ms
   - Impact: 2-3x write improvement

3. **Optimize prepared statements** caching
   - Pre-prepare common queries
   - LRU eviction for unused statements
   - Impact: 1.3-1.5x query improvement

4. **Add read-through cache** for redb
   - Cache hot episodes
   - LRU eviction
   - Impact: 5-10x read improvement for cache hits

**Deliverables:**
- Compression configuration (do-memory-storage-turso/src/compression.rs)
- Batch write operations (do-memory-storage-turso/src/storage/batch/)
- Prepared statement cache (do-memory-storage-turso/src/prepared.rs)
- Read-through cache (do-memory-storage-redb/src/cache.rs)

### Phase 3: Algorithmic Optimization (Week 5-6) - **Expected: 2.5-3x improvement**

**Priority: Medium** | **Risk: Medium** | **Effort: High**

1. **Implement incremental pattern extraction**
   - Track changed episodes
   - Extract only new patterns
   - Cache extraction results
   - Impact: 2-3x pattern extraction improvement

2. **Parallelize extraction** with Rayon
   - Process chunks in parallel
   - Use SIMD for similarity
   - Impact: 2-4x extraction improvement

3. **Optimize DBSCAN clustering**
   - Use spatial indexing
   - Sample for large datasets
   - Impact: 1.5-2x clustering improvement

4. **Implement hybrid search** (FTS5 + vector)
   - Use FTS5 for keyword filtering
   - Vector search on filtered set
   - Impact: 2-3x retrieval improvement

**Deliverables:**
- Incremental pattern extractor (do-memory-core/src/extraction/incremental.rs)
- Parallel extraction pipeline (do-memory-core/src/extraction/parallel.rs)
- Optimized DBSCAN (do-memory-core/src/patterns/dbscan_opt.rs)
- Hybrid search (do-memory-core/src/semantic/hybrid.rs)

### Phase 4: Concurrency & Scaling (Week 7-8) - **Expected: 3-4x improvement**

**Priority: Medium** | **Risk: High** | **Effort: High**

1. **Replace Mutex with DashMap**
   - Sharded concurrent map
   - Better multi-core scaling
   - Impact: 2-3x concurrent ops improvement

2. **Implement lock-free cache**
   - Atomic operations
   - Sharded LRU
   - Impact: 1.5-2x cache improvement

3. **Add work-stealing queue**
   - For async pattern extraction
   - Better load balancing
   - Impact: 1.5-2x throughput improvement

4. **Optimize cache line usage**
   - Pad structures to cache line size
   - Reduce false sharing
   - Impact: 10-15% improvement

**Deliverables:**
- DashMap-based storage (do-memory-core/src/memory/concurrent.rs)
- Lock-free cache (do-memory-core/src/retrieval/atomic_cache.rs)
- Work-stealing queue (do-memory-core/src/learning/work_stealing.rs)
- Cache-line optimized structures (do-memory-core/src/memory/padded.rs)

## Performance Monitoring

### Metrics to Track

```rust
// Add to existing monitoring module
pub struct PerformanceMetrics {
    // Latency metrics (P50, P95, P99)
    pub episode_creation_latency: Histogram,
    pub step_logging_latency: Histogram,
    pub episode_completion_latency: Histogram,
    pub pattern_extraction_latency: Histogram,
    pub memory_retrieval_latency: Histogram,
    
    // Throughput metrics
    pub episodes_per_second: Gauge,
    pub steps_per_second: Gauge,
    pub patterns_per_second: Gauge,
    
    // Cache metrics
    pub cache_hit_rate: Gauge,
    pub cache_miss_rate: Gauge,
    
    // Storage metrics
    pub turso_write_latency: Histogram,
    pub redb_write_latency: Histogram,
    pub compression_ratio: Gauge,
    
    // Concurrency metrics
    pub lock_contention: Gauge,
    pub worker_queue_depth: Gauge,
}

impl PerformanceMetrics {
    pub fn export_prometheus(&self) -> String {
        // Export metrics for Prometheus scraping
    }
}
```

### Benchmarking Strategy

1. **Unit benchmarks** for each optimization
   - Use criterion for microbenchmarks
   - Compare before/after
   - Track regression

2. **Integration benchmarks** for full lifecycle
   - Episode creation → completion
   - Concurrent operations
   - Scaling tests

3. **Load testing** for production readiness
   - Sustained load (1000 eps)
   - Spike testing (10,000 eps)
   - Soak testing (24 hours)

## Risk Assessment

### High Risk

1. **DashMap migration** - May introduce subtle bugs
   - **Mitigation**: Comprehensive testing, gradual rollout
   
2. **Lock-free cache** - Complex to implement correctly
   - **Mitigation**: Use proven libraries (crossbeam), extensive testing

3. **SIMD optimizations** - CPU-specific, may not work everywhere
   - **Mitigation**: Runtime CPU detection, fallback implementations

### Medium Risk

1. **Batch writes** - May increase latency for individual operations
   - **Mitigation**: Configurable batch size, flush on demand

2. **Compression** - May increase CPU usage
   - **Mitigation**: Compress only large payloads, monitor CPU

3. **Incremental extraction** - May miss patterns if tracking fails
   - **Mitigation**: Periodic full extraction, validation

### Low Risk

1. **Connection pooling** - Well-established pattern
2. **Query caching** - Standard optimization
3. **Parallel operations** - Tokio handles well

## Rollback Plan

Each phase can be rolled back independently:

1. **Feature flags** for all optimizations
2. **A/B testing** with gradual rollout
3. **Metrics-driven** rollback criteria
4. **Automated testing** before deployment

## Success Criteria

### Phase 1 Success
- [ ] Episode creation: <25ms P95 (50% improvement)
- [ ] Step logging: <8ms P95 (33% improvement)
- [ ] Episode completion: <250ms P95 (34% improvement)
- [ ] Memory retrieval: <50ms P95 (33% improvement)
- [ ] Zero regressions in other operations

### Phase 2 Success
- [ ] Episode creation: <20ms P95 (43% improvement)
- [ ] Step logging: <5ms P95 (58% improvement)
- [ ] Episode completion: <200ms P95 (47% improvement)
- [ ] Memory retrieval: <40ms P95 (47% improvement)
- [ ] Turso write latency: <8ms P95
- [ ] redb cache hit rate: >80%

### Phase 3 Success
- [ ] Pattern extraction: <500ms P95 (38% improvement)
- [ ] Memory retrieval: <30ms P95 (60% improvement)
- [ ] Episode completion: <150ms P95 (61% improvement)
- [ ] Zero regressions

### Phase 4 Success
- [ ] All targets met (2-3x overall improvement)
- [ ] Concurrent ops: 3x better scaling
- [ ] Lock contention: <5%
- [ ] Cache hit rate: >85%

## Long-term Optimization Opportunities

### Future Enhancements (Beyond v1.0)

1. **WASM optimization**
   - Compile to WebAssembly for portability
   - Use SIMD128 for vectors
   - Potential: 2-3x improvement

2. **GPU acceleration**
   - Offload embedding similarity to GPU
   - Use CUDA/OpenCL for batch operations
   - Potential: 5-10x improvement for large batches

3. **Distributed caching**
   - Redis/Memcached for multi-instance deployments
   - Consistent hashing for cache distribution
   - Potential: Linear scaling

4. **Machine learning optimization**
   - Learn optimal batch sizes
   - Predict cache eviction
   - Adaptive compression levels
   - Potential: 10-20% improvement

5. **Custom storage engine**
   - Rely less on external databases
   - Optimize for our access patterns
   - Potential: 2-5x improvement

## Appendix

### A. Configuration Examples

```toml
# performance-config.toml

[episode_creation]
batch_enabled = true
batch_size = 10
lazy_validation = true
parallel_writes = true

[step_logging]
buffer_size = 100
flush_interval_ms = 100
compression_threshold = 1024

[episode_completion]
parallel_extraction = true
async_embedding = true
incremental_patterns = true

[pattern_extraction]
parallel = true
simd_enabled = true
cache_enabled = true
sample_threshold = 1000

[memory_retrieval]
hybrid_search = true
query_cache_size = 10000
query_cache_ttl_secs = 3600
batch_similarity = true

[storage]
turso_pool_min = 2
turso_pool_max = 16
turso_compression = "zstd"
turso_compression_threshold = 1024

redb_batch_size = 100
redb_cache_size = 10000
redb_cache_ttl_secs = 1800

[embeddings]
batch_size = 100
cache_size = 10000
local_fallback = true
simd_similarity = true

[concurrency]
use_dashmap = true
cache_shards = 16
work_stealing = true
```

### B. Benchmark Results

Results will be updated as optimizations are implemented.

### C. References

- [Tokio Performance Guide](https://tokio.rs/tokio/topics/performance)
- [Rayon Parallelism](https://docs.rs/rayon/)
- [DashMap Documentation](https://docs.rs/dashmap/)
- [SIMD in Rust](https://rust-lang.github.io/packed_simd/)
- [Criterion Benchmarks](https://bheisler.github.io/criterion.rs/book/)

---

**Document Version**: 1.0  
**Last Updated**: 2026-02-12  
**Next Review**: 2026-02-19 (after Phase 1 completion)
