{% extends "base.html" %} {% block title %}Imagination — Maxim Docs | Real-Time Entity Design{% endblock %} {% block meta_description %}How Maxim's imagination system designs SEM entities in real-time when the agent encounters novel objects, creatures, or NPCs in narration.{% endblock %} {% block meta_keywords %}Maxim imagination, entity design, novel percepts, Default Network, arousal gating, ComponentIndex, EntityDesigner, ephemeral registration, SEM components, cognitive architecture{% endblock %} {% block meta_author %}Maxim Project{% endblock %} {% block og_site_name %}Maxim{% endblock %} {% block og_type %}article{% endblock %} {% block structured_data %}
{% endblock %} {% block content %}
← Back to Overview

## The Concept

### Biological Inspiration

Imagination fires during low-arousal idle states — the same way you don't daydream while fighting. When the brain encounters something unfamiliar, it constructs a mental model from prior experience to reason about the novel entity before physically interacting with it. The Default Network, which activates during rest and mind-wandering, gates this process.

When the agent encounters a novel entity mentioned in percept text that has no existing SEM component, the imagination system designs one in real-time. The result is a fully functional entity with sensors, affordances, and failure modes — registered ephemerally for the session as an observable scene entity, sensable via `sense` / `sense_presence`.

#### Why This Matters

Without imagination, the agent can only sense entities that were pre-authored as YAML components. Narration that mentions a "rusted padlock" or "crystal chandelier" would have no sensors or failure modes — the agent could only talk about them. Imagination closes this gap by designing components on-the-fly, making novel entities observable and substrate-encodable.

#### Implementation history

The core pipeline (I1 entity extraction + I2 EntityDesigner + I3 scene-scoped tools) shipped in v0.7.0. Post-0.7.0 additions: archetype scaffolding — the designer infers a structural archetype (humanoid, creature, weapon…) and merges its body-part vocabulary into the LLM-generated spec, reducing hallucinated modulator names on common entity categories; manifest pre-trigger (`process_manifest()`) — called by the orchestrator before the first sim turn to pre-instantiate entities from the scene description, bypassing the mention threshold and arousal gate; and confirmed observe-only scene registration — imagined entities land in the EntityMap as observe-only, sensable via `sense`/`sense_presence` but with no directly callable affordance tools (the agent interacts via its own body tools).

## The Pipeline

Imagination Pipeline Percept Text ↓ Entity Extraction (NLP heuristics) ↓ ImaginationCache check (already imagined this session?) ↓ ComponentIndex Two-Layer Lookup ↓ ├── Match found → Skip (already known) ↓ └── No match → Mention Counter ↓ Count ≥ threshold (default 2)? ↓ DN Arousal Gate (low arousal only) ↓ Energy Budget Check (≥10% remaining) ↓ EntityDesigner LLM Call ↓ Quick Validation (SEM protocol) ↓ register_ephemeral() + ComponentIndex.add() ↓ Scene EntityMap Registration (observe-only)

Each stage is a gate. If any gate fails, the pipeline short-circuits gracefully — the agent falls back to verbal-only interaction with the entity.

## Entity Extraction

Lightweight NLP heuristics extract entity-like noun phrases from narration text. No external model required — this runs on pure string processing.

### Extracted

Physical objects, creatures, weapons, environmental features, items, vehicles, NPCs

Examples: "rusty padlock", "crystal chandelier", "ancient tome", "iron golem"

### Filtered Out

Abstract concepts, body parts, clothing, emotions, time references, generic pronouns

Examples: "courage", "left arm", "leather boots", "dread", "morning"

Two strategies work in parallel:

- Sentence-level intro patterns — catches "You see a rusty gate", "A massive golem blocks the path", "There is a glowing orb"
- Head-noun scanning — matches against a curated indicator vocabulary of entity-like words (weapon, creature, door, chest, etc.)

## ComponentIndex: Two-Layer Lookup

Before imagining anything, each candidate phrase is checked against the `ComponentIndex` to see if an existing component already covers it:

### Layer 1: Alias Table (O(1))

Exact match against component names and declared synonyms from the `component.synonyms` YAML field.

"sword" → weapons/rusty_sword

### Layer 2: Embedding Similarity

Cosine similarity against all component signature embeddings. Threshold: 0.65. Uses the shared `similarity.encoder` singleton.

"old iron door" → environments/rusty_gate (0.72)

If either layer finds a match, imagination is skipped for that phrase. This prevents the system from creating duplicate components under different names.

#### Thread Safety

The ComponentIndex is protected by an RLock. Multiple threads (AUT + orchestrator) can query it concurrently. Persistence uses `.npy` + `.json` sidecar — no pickle, ever.

## Gates

Three gates prevent imagination from firing at inappropriate times:

🔢

#### Mention Threshold

Default 2 mentions before triggering. A one-off phrase ("you notice a crack in the wall") won't spawn an entity. Repeated mentions signal narrative importance.

💤

#### DN Arousal Gate

Only fires during low-arousal idle states. Blocked when the Default Network is inhibited or recent interesting events occurred. You don't daydream while fighting.

🔋

#### Energy Budget

Skipped when LLM energy is critical (<10% remaining). Falls back gracefully to verbal-only interaction with the entity.

#### Per-Phrase Design Guard

A per-phrase lock prevents concurrent LLM calls for the same entity phrase. In multi-thread setups (AUT + orchestrator), only one thread designs a given entity; the other waits for the result. Thread-safe throughout via RLock.

## EntityDesigner

When all gates pass, the `ImaginationDesigner` wraps the `EntityDesigner` and makes a single LLM call to generate a complete SEM component specification from the entity phrase and surrounding narrative context.

LLM Input Entity phrase: "crystal chandelier" Narrative context: "A massive crystal chandelier hangs from the vaulted ceiling, its facets catching the torchlight." Genre: fantasy Existing components in scope: [rusty_sword, base_humanoid, ...] LLM Output (validated SEM spec) entity: name: crystal_chandelier entity_type: environmental_feature sensors: stability: {unit: ratio, range: [0, 1], initial: 0.9} illumination: {unit: ratio, range: [0, 1], initial: 0.7} crystal_count: {unit: count, range: [0, 50], initial: 42} modulators: swing: {params: {force: float}, timeout: 3} shatter_crystal: {params: {target: string}, timeout: 2} cut_chain: {params: {}, timeout: 5} failure_modes: collapse: {trigger: {field: stability, op: "<", value: 0.1}, pain_intensity: 0.9} darkness: {trigger: {field: crystal_count, op: "<", value: 5}, pain_intensity: 0.2}

### Quick Validation

After generation, the spec is validated against the SEM protocol: required fields present, sensor ranges valid, modulator params typed, failure triggers well-formed. Invalid specs are discarded — the agent falls back to verbal interaction.

## Ephemeral Registration

Imagined entities live in a separate overlay (`_ephemeral_index`) from the persistent component registry. This separation is architectural:

### During Session

- Visible to `get()`, `has()`, `query()`
- Registered as observe-only scene entity (sensable via `sense` / `sense_presence`)
- Added to ComponentIndex for dedup
- Affordance metadata present for substrate encoding; agent interacts via its own body tools

### At Session End

- Cleared via `clear_ephemeral()`
- Tools deregistered
- Episodes persist (with provenance)
- Causal links get 50% confidence decay

This means the agent learns from imagined interactions (pain avoidance, reward prediction) but with reduced confidence, reflecting the simulated origin.

## Provenance Tagging

All learning from imagined entities carries `imagined=True` provenance:

Provenance Flow Episode from imagined entity interaction: episode.metadata["imagined"] = True episode.metadata["imagined_entity"] = "crystal_chandelier" CausalLink from imagined entity outcome: link.metadata["imagined"] = True On session end (entity discard): NAc.decay_imagined_links(factor=0.5) → All links with imagined=True get confidence *= 0.5

The 50% decay means the agent retains partial learning ("chandeliers can collapse") but with appropriately reduced confidence compared to verified real-world interactions. If the agent encounters the same entity type again and the interaction confirms the learned pattern, confidence rebuilds naturally through standard Rescorla-Wagner updates.

## Scene Entity Registration

Imagined entities are registered as observe-only scene entities in the EntityMap. The agent can sense them but cannot call their affordances directly as tools. The agent interacts using its own body’s tools (the self-entity), not the scene entity’s affordances. This means:

- Sensable via `sense` and `sense_presence` tools
- Affordance names are encoded through the substrate path for concept transfer (EC → ATL → NAc eligibility)
- The scene-scoped tool cap (I3) applies to the agent’s own body tools, not to imagined scene entities

This integrates with the scene-scoped tool system — imagined entities follow the same observe-only lifecycle as pre-authored scene entities.

## ImaginationCache

A session-scoped cache prevents redundant design attempts:

- Shared across AUT + orchestrator — if the orchestrator's narration mentions "crystal chandelier" and the AUT's perception also extracts it, only one design call fires
- Stores both successes and failures — a failed validation for "vague smoke" won't retry every turn
- Thread-safe via RLock
- Cleared at session end alongside the ephemeral registry

## Integration with Bio-Systems

Imagined entities participate in the full bio-pipeline, just like pre-authored ones:

🧠

#### Hippocampus

Episodes from imagined interactions are captured with `imagined=True` metadata.

🎯

#### NAc

Causal links form from affordance outcomes. 50% confidence decay at session end.

⚡

#### PainBus

Failure modes fire pain signals through the same cascade as real entities.

🧩

#### Cerebellum

Forward models train on imagined affordance outcomes via Rescorla-Wagner.

📚

#### ATL

Semantic concepts form from imagined entity interactions (modality-tagged).

🎭

#### Acting Coach

Exploration directives include imagined entity affordances in the meta-prompt.

## Architecture

Module | Purpose
--- | ---
imagination/trigger.py | Entity noun-phrase extraction, ComponentIndex lookup, design dispatch
imagination/designer.py | ImaginationDesigner: wraps EntityDesigner for real-time entity generation
imagination/cache.py | Session-scoped ImaginationCache, thread-safe, shared AUT + orchestrator
embodiment/component_registry.py | register_ephemeral(), clear_ephemeral(), ephemeral overlay
embodiment/component_index.py | Two-layer semantic discovery (alias hash + embedding cosine)
tools/registry.py | Scene-scoped activation, active tool cap, executor gate
runtime/agent_loop.py | imagination_trigger parameter on run_agentic_loop

### Wiring

The imagination trigger is passed as a parameter to `run_agentic_loop` and fires post-`state.update()` on every turn. This placement means the agent has already processed the percept and updated its state before imagination considers whether to design a new entity.

Agent Loop Integration while running: percept = await percept_source.next() state.update(percept) # Imagination fires here — after state update, before decision if imagination_trigger: imagination_trigger.process(percept.text, scene_context) decision = await exec_agent.decide(state) result = await executor.execute(decision)

## Related Systems

### Embodiment (SEM Protocol)

The entity format imagination generates. Sensors, modulators, failure modes.

### Component Library

80+ pre-authored entities. ComponentIndex searches these before imagining.

### Asset Foundry

Batch entity generation with gauntlet testing. Imagination is the real-time equivalent.

### Scene-Scoped Tools

How imagined entity tools activate/deactivate with scene changes.

← Previous Embodiment Next → Component Library

### All Chapters

Overview The Agent Brain Memory Systems Math Cognition Attention & Salience Body Awareness Embodiment Imagination Component Library Communication Usage Guide Technical Deep Dive Operating Modes Networking Simulation Benchmarks Roadmap
{% include "_maxim_footer.html" %}
{% endblock %}