# Memory System

Maxim's memory system spans multiple bio-inspired subsystems, each handling a different kind of knowledge.

## Overview

| Subsystem | Brain Region | What It Stores |
|-----------|-------------|----------------|
| **Hippocampus** | Hippocampus | Episodic memories (events, conversations) with associative indexing |
| **ATL** | Anterior Temporal Lobe | Semantic concepts, entities, and typed relationships |
| **Angular Gyrus** | Angular Gyrus / IPS | Mathematical cognition, cross-modal statistical summaries |
| **NAc** | Nucleus Accumbens | Causal links and reward prediction errors |
| **SCN** | Suprachiasmatic Nucleus | Temporal rhythm indexing for time-based retrieval |
| **EC** | Entorhinal Cortex | Multi-modal similarity routing with Phase 4 neural embeddings |
| **MemoryHub** | (integration layer) | Cross-system coordinator, event dispatch, lifecycle management |

## Components

| Component | File | Purpose |
|-----------|------|---------|
| `Hippocampus` | `hippocampus.py` | Core episodic memory graph |
| `StateStore` | `state_store.py` | State snapshot caching |
| `RWLock` | `rwlock.py` | Reader-writer lock for concurrent access |
| `MemoryStrategy` | `strategies.py` | Retention scoring algorithms |

---

## Hippocampus

The Hippocampus is the central memory store for episodic memories.

### Memory Types

| Type | Class | Contents |
|------|-------|----------|
| Perception | `Perception` | Sensory input (vision, audio) |
| Action | `Action` | Executed actions and parameters |
| Decision | `Decision` | Planning decisions and rationale |
| Outcome | `Outcome` | Action results and feedback |
| Context | `Context` | Environmental state |

### Capturing Memories

```python
from maxim.memory import Hippocampus, HippocampusConfig, Perception

config = HippocampusConfig(
    max_nodes=10_000,
    persistence_path="~/.maxim/memory/hippocampus.json",
    indexed_keys=frozenset({"goal", "tool", "object", "person"}),
)

hippo = Hippocampus(config)

# Capture a perception
perception = Perception(
    observations={"scene": "office", "confidence": 0.95},
    cli_input=None,
    transcript=None,
    detected_objects=["person", "book"],
    detected_people=["alice"],
    salience=0.8,
    novelty=0.6,
)
memory_id = hippo.capture(perception)
```

### Querying Memories

```python
# Recall by filters (goal, tool, success, mode, time range)
memories = hippo.recall(goal="find_book")

# Recall by time range (parameters on recall())
recent = hippo.recall(
    time_after=time.time() - 3600,  # Last hour
    time_before=time.time(),
)

# Get by ID (O(1))
memory = hippo.get(memory_id)

# Similarity-based recall (perception overlap)
similar = hippo.recall_similar(
    perception=current_perception,
    limit=5,
    include_compressed=True,
)

# Associative recall (spreading activation through graph)
associated = hippo.recall_associated(
    seed_ids=[memory_id],
    limit=10,
    decay=0.5,
    max_depth=3,
    threshold=0.05,
    include_compressed=True,
)
# Returns list of (memory, activation_score) tuples sorted by score
```

### Configuration

```python
@dataclass
class HippocampusConfig:
    max_nodes: int = 10_000
    state_store_max_entries: int = 1000
    persistence_path: str | None = None
    indexed_keys: frozenset[str] = frozenset({"goal", "tool", "object", "person", "success", "mode"})

    # Sleep consolidation
    enable_sleep_consolidation: bool = True
    max_age_without_access: float = 7 * 24 * 3600  # 1 week
    compression_age: float = 24 * 3600             # 1 day
    retention_threshold: float = 0.3
    compression_threshold: float = 0.6
    memory_strategy: str = "access_based"

    # Long-term memory consolidation
    consolidate_during_sleep: bool = True
    max_consolidation_candidates: int = 500
    immediate_promotion_salience: float = 0.95
    immediate_promotion_novelty: float = 0.95
    long_term_retention_boost: float = 2.0
    auto_save_after_sleep: bool = True

    # Associative graph
    enable_associative_graph: bool = True
    association_limit: int = 5
    association_threshold: float = 0.5
    spreading_activation_decay: float = 0.5
    spreading_activation_max_depth: int = 3
    spreading_activation_threshold: float = 0.05

    # Async capture
    capture_queue_size: int = 100
```

---

## Memory Strategies

Retention strategies determine which memories to keep, compress, or remove.

### Available Strategies

| Strategy | Key Factor | Use Case |
|----------|------------|----------|
| `AccessBasedStrategy` | Recency + frequency of access | General use |
| `ImportanceBasedStrategy` | Salience + novelty scores | High-value retention |
| `TemporalAwareStrategy` | Age-weighted scoring | Decay over time |
| `CompositeStrategy` | Weighted combination | Custom policies |

### Usage

```python
from maxim.memory import (
    AccessBasedStrategy,
    ImportanceBasedStrategy,
    CompositeStrategy,
)

# Single strategy
strategy = AccessBasedStrategy()

# Composite with weights
strategy = CompositeStrategy([
    (AccessBasedStrategy(), 0.4),
    (ImportanceBasedStrategy(), 0.6),
])

# Apply to hippocampus
hippo = Hippocampus(config, strategy=strategy)
```

---

## Sleep Consolidation

During sleep mode, the Hippocampus performs memory consolidation via `ConsolidationMixin` (implemented in `memory/hippocampus_consolidation.py`).

### Consolidation Pipeline

```
Agent Cycle → _evaluate_staging() → Acute Staging (sidecar JSON in short_term_memory/)
                                         (significance score via SignificanceWeightLearner)
Sleep Mode → hippocampus.sleep()
                ├── _consolidate(): promote candidates to LONG_TERM
                │    ├── salience > 0.9
                │    ├── novelty > 0.9
                │    ├── user interaction + success
                │    ├── access_count >= 5
                │    └── promotion_pressure >= 3.0 (use-based path)
                ├── Compress: old records → CompressedMemory
                └── Remove: stale, low-score memories
```

### Significance Heuristics

Completed cycles are scored by `SignificanceWeightLearner` (in `exec_agent`) using 6 heuristics:

| Heuristic | Baseline Weight | Signal |
|-----------|----------------|--------|
| `rpe_magnitude` | 0.35 | NAc prediction error |
| `user_interaction` | 0.20 | CLI/transcript present |
| `novelty` | 0.15 | Perception novelty |
| `plan_phase_boundary` | 0.10 | Phase start/complete/fail |
| `energy_state_change` | 0.05 | Crossed low/critical |
| `outcome_valence_extremity` | 0.10 | Very good/bad outcome |

When the weighted score exceeds `staging_threshold` (default 0.5), a JSON sidecar is written to `data/short_term_memory/`.

### API

```python
# Trigger sleep consolidation (compress, remove, promote)
hippo.sleep()

# Or trigger promotion-only consolidation
hippo.consolidate()

# Consolidation runs automatically during sleep mode
# See modes/definitions.py for sleep mode behavior
```

### Compressed Memories

Old memories are compressed to save space:

```python
@dataclass
class CompressedMemory(CompressedRecord):
    """Lightweight summary of an old episodic memory.

    Inherits from CompressedRecord (→ MemoryRecord): id, timestamp,
    created_at, accessed_at, access_count, long_term, consolidated_at,
    edge_count, touch().
    """
    run_id: str = ""

    # Essential decision data (for queries)
    goal: str | None = None
    tool_name: str = ""
    success: bool = False

    # Minimal perception summary
    had_user_input: bool = False
    object_count: int = 0
    novelty: float = 0.5
    salience: float = 0.5
```

---

## StateStore

Caches state snapshots for fast retrieval during planning.

```python
from maxim.memory import StateStore

store = StateStore(max_entries=1000)

# Store a state snapshot and get its hash reference
state = {"position": (45, 10), "objects": ["book", "pen"]}
state_ref = store.store(state)  # Returns 16-char SHA-256 hash, e.g. "a1b2c3d4e5f67890"

# Retrieve by hash reference
cached = store.get(state_ref)  # Returns the dict, or None if evicted

# Duplicate states share a single entry (content-addressed)
same_ref = store.store(state)  # same hash as before

# Other methods
store.contains(state_ref)  # bool
len(store)                 # number of cached entries
store.stats()              # {"entries": ..., "max_entries": ..., "utilization": ...}
store.clear()              # drop all entries
```

---

## Concurrency

The Hippocampus uses RWLock for thread-safe access:

```python
from maxim.memory import RWLock

lock = RWLock()

# Multiple readers
with lock.read_lock():
    memories = hippo.recall(goal="example")

# Exclusive writer
with lock.write_lock():
    hippo.capture(new_memory)
```

---

## Persistence

Hippocampus persists to JSON:

```python
# Save manually
hippo.save("~/.maxim/memory/hippocampus.json")

# Load on init if path exists
hippo = Hippocampus(HippocampusConfig(
    persistence_path="~/.maxim/memory/hippocampus.json"
))

# Auto-save on shutdown
# Handled by runtime shutdown hooks
```

### File Format

```json
{
  "_format_version": "1.0",
  "saved_at": 1707235200.0,
  "memories": [...],
  "context_index": {...},
  "stats": {...},
  "compressed_count": 0,
  "associative_graph": {...},
  "episodes": [...],
  "next_episode_ordinal": 0,
  "node_modality": {}
}
```

Clear with: `maxim --clear-memory hippo`

---

## Similarity Indices (LSH)

Two `SimilarityIndex` instances provide O(1) approximate similarity lookup using MinHash + Locality-Sensitive Hashing:

| Index | Purpose | Used By |
|-------|---------|---------|
| `context_index` | Language/context similarity | `AssociationIndex.find_similar()`, `ConsolidationOrchestrator` |
| `percept_index` | Percept similarity | `recall_deep`, chronic recurrence detection |

```python
from maxim.memory.context_index import SimilarityIndex

index = SimilarityIndex(num_hashes=64, num_bands=8)
index.register("mem_1", "the robot saw a person near the table")
results = index.query_similar("person at the table", min_similarity=0.3)
# Returns [(memory_id, estimated_jaccard_similarity), ...]
```

The LSH index replaces the old keyword Jaccard approach in `AssociationIndex.find_similar()` and O(n) linear scans in `hippocampus.recall_similar()`.

---

## ATL (Anterior Temporal Lobe)

The ATL is the semantic concept memory layer. While the Hippocampus stores *what happened* (episodes), the ATL stores *what things mean* (concepts, entities, relationships).

### Architecture

Concepts are promoted from episodes via NAc reward signals and StatisticianAgent pattern confirmation, or ingested directly (RAG). The ATL pipeline has four stages:

| Stage | Module | Purpose |
|-------|--------|---------|
| Extraction | `concept_extractor.py` | Extract candidate concepts from episodic memories |
| Grounding | `concept_grounder.py` | Validate concepts against perceptual evidence |
| Pattern Completion | `pattern_completer.py` | Fill in missing attributes from partial cues |
| Promotion | `semantic_promoter.py` | Promote recurring patterns to stable semantic concepts |

### Data Model

ATL stores `SemanticMemory` instances (via `MemoryLayer` ABC) with typed relationships between concepts. Compressed forms (`CompressedSemantic`) reduce storage for infrequently accessed concepts.

### Integration

- **Hippocampus**: Concepts are derived from episodic memories; ATL wires a `_pattern_completion_fn` back into the Hippocampus for predictive context during memory formation.
- **NAc**: Reward signals gate which patterns are worth promoting to semantic status.
- **AngularGyrus**: Math/statistical enrichment data attached to concepts via `MathContextEntry`.

Clear with: `maxim --clear-memory atl`

---

## Angular Gyrus

Mathematical and statistical cognition. Combines memories across modalities and provides quantitative reasoning. Includes IPS (Intraparietal Sulcus) for fast statistical summaries.

Lives in `src/maxim/math/`. See the Math Cognition guide for details.

The Angular Gyrus does not currently have a dedicated `--clear-memory` key.

---

## MemoryHub

`integration/memory_hub.py` is the central coordinator that wires all 11 bio-systems together. It owns the lifecycle connections between Hippocampus, ATL, NAc, SCN, EC, Salience, Attention, Fear, Spatial, Planning, and Escalation bridges.

### Responsibilities

- Routes perception, decision, action, and outcome events to the correct subsystems
- Manages session start/end lifecycle (including Hippocampus capture thread shutdown)
- Provides a single `on_cycle()` entry point for the agent loop
- Coordinates sleep consolidation across systems

The MemoryHub does not store memories itself -- it dispatches to the appropriate subsystem.

---

## CrossLayerGraph

`memory/cross_layer.py` maintains associative edges *between* memory systems (e.g., linking an episodic memory in Hippocampus to a semantic concept in ATL). This is distinct from the intra-system associative graph inside Hippocampus.

Edge weights combine perceptual (0.6), goal (0.25), and temporal (0.15) similarity. Spreading activation traverses cross-layer edges with configurable decay and depth.

---

## Store Protocols

`memory/store.py` defines three persistence protocols, one per subsystem:

| Protocol | Subsystem | Query Pattern |
|----------|-----------|---------------|
| `EpisodicStore` | Hippocampus | Similarity search, time-range queries |
| `CausalStore` | NAc | Event-to-outcome lookups |
| `SemanticStore` | ATL | Concept-type filtering |

Default implementations (`FileEpisodicStore`, `FileCausalStore`, `FileSemanticStore`) wrap JSON persistence. Database implementations (PostgreSQL + pgvector) are provided by the `[database]` extra for the [Maxim Oasis](hivemind.md) — substrate-primary instances that absorb contributions from many users. (The "Mother Maxim" plan was superseded 2026-05-09 by [Maxim Hivemind + Oasis](hivemind.md); the file-based portable substrate-snapshot bundle is the new primary exchange format.)

---

## NAc (Nucleus Accumbens)

Not strictly a memory system, but tightly coupled. The NAc learns causal links (cause-and-effect relationships) from episodic sequences. It maintains reward prediction errors (RPE) that drive memory consolidation thresholds and concept promotion in the ATL.

Lives in `src/maxim/decisions/nac.py`. See [decisions.md](decisions.md) for full documentation.

Clear with: `maxim --clear-memory nac`

---

## SCN (Suprachiasmatic Nucleus)

The internal clock. Provides temporal rhythm indexing so memories can be queried by time-of-day patterns and circadian-like cycles.

Lives in `src/maxim/time/`. See [time.md](time.md) for full documentation.

Clear with: `maxim --clear-memory scn`

---

## Integration Points

| System | Integration |
|--------|-------------|
| **SCN** | Temporal indexing for time-based queries |
| **NAc** | Causal learning from episodic sequences |
| **ConsolidationMixin** | Sleep consolidation: promote, compress, remove (`hippocampus_consolidation.py`) |
| **SignificanceWeightLearner** | Learns which heuristics predict useful memories |
| **SimilarityIndex (LSH)** | O(1) context/percept similarity for recall and consolidation |
| **SalienceMemoryBridge** | Updates salience from memory patterns |
| **SpatialMemoryBridge** | Enriches spatial map with memory |
| **EC Similarity** | Phase 4 neural semantic embeddings for deep similarity queries |

### Semantic Embedding (Phase 4)

When semantic similarity is enabled, the EC uses neural embeddings for deep semantic understanding:

```python
from maxim.similarity import ECConfig, EntorhinalCortex

# Enable Phase 4 semantic embeddings
ec = EntorhinalCortex(ECConfig(
    enable_semantic=True,
    semantic_model="all-MiniLM-L6-v2",  # 80MB model
    async_embedding=True,  # Non-blocking capture
))

# Semantic queries work across synonyms
results = ec.find_semantic("find the coffee mug", k=10)
# "cup" memories will match with high similarity
```

Semantic embeddings are automatically generated when memories are captured (via capture callback) and persist across sessions in `~/.maxim/util/semantic_embeddings.npz`.

See [bridges.md](archive/bridges.md) for bridge documentation.

---

## Memory Hierarchy

Maxim maintains multiple memory tiers with different lifespans and retrieval modes:

```
Percept Buffer → Working Notes → Short-Term Memory → Long-Term Episodic → Semantic Concepts
  (prompt)        (prompt)        (MemoryAgent)       (Hippocampus)       (ATL: objects,
                                                                           people, goals,
                                                                           relationships)
```

| Tier | Retrieval | Contents | Lifespan |
|------|-----------|----------|----------|
| Percept buffer | Always present (last N) | Recent percepts, outcomes | Minutes |
| Working notes | Always present (file read) | Deliberately pinned context | Until LLM removes it |
| MemoryAgent working memory | Association + activation | WorkingMemoryEntry wrappers | Session (decays/promotes) |
| Hippocampus (long-term) | Similarity recall | Consolidated episodic memories | Permanent (retention-scored) |
| ATL (semantic) | Concept lookup + activation | Concepts, relationships, causal patterns | Permanent |
| Plan system | Always present when active | Structured goals/phases | Until plan completes |

**Working notes** (`notes/context.md`) give the LLM a persistent scratchpad — always in the prompt, edited via `write_file`. Unlike similarity-based Hippocampus recall, working notes survive regardless of what the LLM is currently perceiving.

**StructuredContext** (built by MemoryAgent each cycle) assembles these tiers into a single object consumed by ExecAgent for goal proposal. Fields include `relevant_memories`, `working_notes`, `workspace_files`, `knowledge_context`, and `plan_progress`.

## WorkingMemoryEntry and Staged Formation

MemoryAgent wraps all memory records in `WorkingMemoryEntry[T]` — a generic
wrapper that holds any `MemoryRecord` subclass plus agent-level metadata
(salience, decay_rate, tier, predicted outcomes).

### Memory Tier Lifecycle

```
FORMING → SHORT_TERM → LONG_TERM → consolidated out
```

- **FORMING**: Created at percept time, filled incrementally during pipeline. Eviction-protected.
- **SHORT_TERM**: Normal decay and eviction. Promotes to LONG_TERM via use-based consolidation (context-diverse recall accumulates `promotion_pressure`; wall-clock decay prevents trickle-promotion; threshold crossing triggers promotion) or during sleep consolidation (high salience/novelty/access count).
- **LONG_TERM**: Age-based eviction. Consolidation marks records in Hippocampus.

Active-reference context (recent percepts, outcomes, speech, CLI inputs) lives in
`WorkingMemorySet` — an Exec-owned layer in `agents/working_memory.py`, not a
memory tier. This was previously the orphaned `MemoryTier.WORKING` which was
removed in 0.8.

### Staged Formation

EpisodicMemory is constructed incrementally and held in the forming pool
throughout the pipeline:

1. **Percept arrives** → `_begin_memory_formation()` creates FORMING entry with Perception+Context
2. **Decision made** → `_update_forming_decision()` fills in Decision
3. **Action executes** → `_update_forming_action()` fills in Action
4. **Outcome received** → `_complete_forming_memory()` fills Outcome, transitions to SHORT_TERM
5. **New cycle** → `_flush_completed_from_pool()` removes completed entries from the forming pool

### Pattern Completion Hook

Optional `_pattern_completion_fn` callable (wired by ATL) provides predictive
context during FORMING. Returns `list[PredictedOutcome]` with typed fields
(`tool`, `success`, `goal`, `confidence`, `math_context`, `source_episode_id`).

### Type Contracts

- `PredictedOutcome`: Typed prediction from graph chaining (defined in `types.py`)
- `MathContextEntry`: Per-concept math enrichment data (defined in `types.py`)
- `WorkingMemoryEntry[T]`: Generic wrapper (defined in `bus.py`)

### Coexistence with Hippocampus

MemoryAgent and Hippocampus maintain separate EpisodicMemory instances.
`agent_loop.py` continues calling `capture_from_loop_async()` independently.
The two systems coexist — MemoryAgent gets structured types in working memory,
Hippocampus retains all capture hooks (associations, consolidation, promotion).

## Memory Flow

```
Perception/Action/Decision
          ↓
    Hippocampus.capture()
          ↓
    ┌─────┴─────┐
    │  Indexing │
    ├───────────┤
    │ Hash keys │ → O(1) lookup
    │ SCN bins  │ → Temporal query
    │ EC embed  │ → Semantic search
    └───────────┘
          ↓
    StateStore (cache)
          ↓
    Sleep Consolidation (hippocampus.sleep())
    ├── _consolidate(): promote candidates to LONG_TERM
    │    └── criteria: salience/novelty > 0.9, user+success, access_count >= 5,
    │                  promotion_pressure >= 3.0
    ├── Compress: old records → CompressedMemory
    └── Remove: stale, low-score memories
```
