---
title: "Memory and compaction"
description: "Automatic context compaction and semantic memory for long-running conversations"
icon: "brain"
---

Two systems handle conversations that exceed model context limits:

1. **Context compaction** -- automatically summarizes conversation history when the context window fills up, preserving recent turns and discarding older ones.
2. **Semantic memory** -- indexes discarded messages so the agent can retrieve past context on demand via the `memory_search` tool.

Together they allow agents to maintain coherent multi-turn sessions that exceed any single model's context limit.

## What this guide is for

Use this guide when you want to understand or configure:

- long-horizon conversation behavior
- semantic recall via `memory_search`
- compaction thresholds
- the relationship between compaction and semantic memory

## Feature flags

Compaction and semantic memory have separate build-time and runtime switches.

| Feature flag | Cargo feature | What it enables |
|---|---|---|
| Compaction strategy | `session-compaction` | `DefaultCompactor` in `meerkat-session` |
| Memory store backend | `memory-store` | CLI convenience feature enabling the memory store backend and session wiring |
| Memory session wiring | `memory-store-session` | `HnswMemoryStore`, `memory_search`, and memory integration with the agent/session path |

<Note>
The CLI's default features include `skills` but **not** `session-compaction` or memory features. You must enable them explicitly for source builds.
</Note>

```bash
cargo build -p rkat --features session-compaction,memory-store
```

At runtime, `AgentFactory::memory(true)` enables the semantic memory path (`HnswMemoryStore` + `memory_search`). Compaction is controlled independently by whether the binary is built with `session-compaction` and whether a compactor is wired into the session path.

```rust
let factory = AgentFactory::new(store_path)
    .builtins(true)
    .memory(true);  // enable semantic memory

// Per-request override:
let mut build_config = AgentBuildConfig::new("claude-sonnet-4-6");
build_config.override_memory = ToolCategoryOverride::Enable;
```

That means these cases are all valid:

- compaction enabled, semantic memory disabled
- semantic memory enabled, compaction unavailable
- both enabled together

## CompactionConfig

Controls when and how compaction runs.

| Field | Type | Default | Description |
|---|---|---|---|
| `auto_compact_threshold` | `u64` | `100_000` | Compaction triggers when `last_input_tokens >= threshold` OR `estimated_history_tokens >= threshold`. |
| `recent_turn_budget` | `usize` | `4` | Number of recent complete turns to retain after compaction. A turn is a User message followed by all subsequent non-User messages until the next User message. |
| `max_summary_tokens` | `u32` | `4096` | Maximum tokens the LLM may produce for the compaction summary. |
| `min_turns_between_compactions` | `u32` | `3` | Minimum turns that must elapse between consecutive compactions (loop guard to prevent runaway compaction). |

## How compaction triggers

Compaction is checked at every turn boundary, just before the next LLM call.

The decision flow:

<Steps>
  <Step title="Skip turn 0">
    The first turn always skips compaction.
  </Step>
  <Step title="Loop guard check">
    If compaction occurred at turn N, no compaction until turn N + `min_turns_between_compactions`.
  </Step>
  <Step title="Dual threshold evaluation">
    Compaction triggers if EITHER:
    - `last_input_tokens >= auto_compact_threshold` (input tokens from the last LLM response), OR
    - `estimated_history_tokens >= auto_compact_threshold` (JSON bytes of all messages / 4).
  </Step>
</Steps>

## What happens during compaction

When compaction triggers:

<Steps>
  <Step title="Emit CompactionStarted event">
    Emitted with input/estimated token counts and message count.
  </Step>
  <Step title="Send compaction prompt to LLM">
    The current conversation history plus a compaction prompt is sent to the LLM with no tools and `max_summary_tokens` as the response limit.
  </Step>
  <Step title="Handle result">
    **On failure**: a CompactionFailed event is emitted and the session is not mutated (safe failure).

    **On success**: `DefaultCompactor::rebuild_history` produces new messages:
    - System prompt is preserved verbatim (if present).
    - A summary message is injected as a User message with the prefix `[Context compacted]`.
    - The last `recent_turn_budget` complete turns are retained.
    - All other messages become `discarded`.
  </Step>
  <Step title="Index discarded memory">
    If semantic memory is enabled, discarded messages are indexed before the compacted history is committed.
  </Step>
  <Step title="Replace session messages">
    The session messages are replaced with the rebuilt history after memory indexing accepts the discarded content.
  </Step>
  <Step title="Record usage and emit completion event">
    Compaction usage is recorded against the session and budget. A **CompactionCompleted** event is emitted with summary token count and before/after message counts.
  </Step>
</Steps>

<Accordion title="The compaction prompt">
The compactor sends this prompt to the LLM:

> You are performing a CONTEXT COMPACTION. Your job is to create a handoff summary so work can continue seamlessly.
>
> Include:
> - Current progress and key decisions made
> - Important context, constraints, or user preferences discovered
> - What remains to be done (clear next steps)
> - Any critical data, file paths, examples, or references needed to continue
> - Tool call patterns that worked or failed
>
> Be concise and structured. Prioritize information the next context needs to act, not narrate.
</Accordion>

## Memory indexing after compaction

When both a `Compactor` and a `MemoryStore` are wired into the agent, discarded messages are indexed into semantic memory before compacted history is committed. If the memory store rejects indexing, Meerkat preserves the original history, emits a `CompactionFailed` event, and skips that compaction attempt instead of dropping the only authoritative copy of the discarded text.

For each discarded message:
- The message's indexable text content is extracted via `message.as_indexable_text()`.
- If non-empty, it is indexed with `MemoryMetadata` containing the session ID, current turn number, and a timestamp.

This means previously discarded conversation content becomes searchable via the `memory_search` tool.

## The `memory_search` tool

When memory is enabled, the agent gains a `memory_search` tool.

### Tool definition

| Property | Value |
|---|---|
| Name | `memory_search` |
| Description | Search semantic memory for past conversation content. Memory contains text from earlier conversation turns that were compacted away to save context space. |

### Parameters

<ParamField path="query" type="string" required>
  Natural language search query describing what you want to recall.
</ParamField>

<ParamField path="limit" type="integer" default="5">
  Maximum number of results to return. Capped at 20.
</ParamField>

### Response format

Returns a JSON array of result objects:

```json
[
  {
    "content": "The text content of the memory entry",
    "score": 0.85,
    "session_id": "019467d9-7e3a-7000-8000-000000000000",
    "turn": 3
  }
]
```

<ResponseField name="content" type="string">
  The text content of the memory entry.
</ResponseField>

<ResponseField name="score" type="number">
  Similarity score from 0.0 (no match) to 1.0 (exact match). Typical useful matches are above 0.7.
</ResponseField>

<ResponseField name="session_id" type="string">
  The session the memory originated from (enables cross-session recall).
</ResponseField>

<ResponseField name="turn" type="integer">
  The turn number within the session when the memory was indexed.
</ResponseField>

## Memory store implementations

<Tabs>
  <Tab title="HnswMemoryStore (production)">
    Uses:
    - **hnsw_rs** (v0.3) for approximate nearest-neighbor search with cosine distance.
    - **SQLite** for persistent metadata and text storage.

    Storage layout: `.rkat/memory/memory.sqlite3`

    Key characteristics:
    - **Embedding**: Bag-of-words TF with hash-based dimensionality reduction (4096-dimensional vectors, L2-normalized). Each word is hashed to a bucket and its presence increments that dimension.
    - **Persistence**: Data survives process restart. On `open()`, all existing entries are re-indexed into the HNSW graph from SQLite.
    - **Score conversion**: HNSW cosine distance (0 = identical, 2 = opposite) is converted to a 0..1 similarity score: `score = 1.0 - (distance / 2.0)`.
    - **Thread safety**: Insertions are serialized via a `Mutex` to couple point ID allocation with successful writes. Searches use a `RwLock` for concurrent reads.
    - **Constants**: `MAX_NB_CONNECTION = 16`, `MAX_LAYER = 16`, `EF_CONSTRUCTION = 200`, `DEFAULT_MAX_ELEMENTS = 100_000`.
  </Tab>
  <Tab title="SimpleMemoryStore (test-only)">
    In-memory substring matching -- not suitable for production.

    - Stores entries in a `Vec<MemoryEntry>` behind a `RwLock`.
    - Search: lowercases both query and content, counts matching words, scores as `matching_words / total_query_words`.
    - No persistence -- data is lost when the process exits.
  </Tab>
</Tabs>

## How memory gets wired

When the `memory-store-session` feature is compiled in and memory is enabled:

1. An `HnswMemoryStore` is opened at `{store_path}/memory/`.
2. The `memory_search` tool is added to the agent's tool set.
3. A `DefaultCompactor` is attached only if `session-compaction` is also enabled.
4. A built-in `memory-retrieval` skill is injected into the system prompt, teaching the agent how to use memory search.

## Examples

<Tabs>
  <Tab title="CLI">
    ```bash
    # Build with memory + compaction features
    cargo build -p rkat --features session-compaction,memory-store

    # Enable the richer tool preset so memory_search is available
    rkat run --tools full "Analyze this large codebase..."
    ```
  </Tab>
  <Tab title="SDK">
    ```rust
    use meerkat::{AgentFactory, AgentBuildConfig};
    use meerkat_store;

    let realm = meerkat_store::realm_paths("team-alpha");
    let factory = AgentFactory::new(realm.root.clone())
        .runtime_root(realm.root)
        .project_root(".")
        .builtins(true)
        .memory(true);

    let mut build_config = AgentBuildConfig::new("claude-sonnet-4-6");
    // Memory + compaction are wired automatically when:
    //   1. The factory has memory(true)
    //   2. The binary is compiled with session-compaction + memory-store-session features

    let agent = factory.build_agent(build_config, &config).await?;
    ```
  </Tab>
</Tabs>

### Custom CompactionConfig

```rust
use meerkat_core::CompactionConfig;
use meerkat_session::DefaultCompactor;

let compactor = DefaultCompactor::new(CompactionConfig {
    auto_compact_threshold: 50_000,   // Compact earlier
    recent_turn_budget: 6,            // Keep more recent turns
    max_summary_tokens: 8192,         // Allow longer summaries
    min_turns_between_compactions: 5, // More spacing between compactions
});
```

## See also

- [Configuration: memory and compaction](/concepts/configuration) - config file settings
- [Architecture](/reference/architecture) - how compaction fits into the agent loop
