MAXIM
Real-Time Entity Design from Novel Percepts
Shipped in v0.7.0 (I1 + I2); evolved through v0.8–0.9
Imagination fires during low-arousal idle states — the same way you don't daydream while fighting. When the brain encounters something unfamiliar, it constructs a mental model from prior experience to reason about the novel entity before physically interacting with it. The Default Network, which activates during rest and mind-wandering, gates this process.
When the agent encounters a novel entity mentioned in percept text that has no existing SEM component, the imagination system designs one in real-time. The result is a fully functional entity with sensors, affordances, and failure modes — registered ephemerally for the session as an observable scene entity, sensable via sense / sense_presence.
Without imagination, the agent can only sense entities that were pre-authored as YAML components. Narration that mentions a "rusted padlock" or "crystal chandelier" would have no sensors or failure modes — the agent could only talk about them. Imagination closes this gap by designing components on-the-fly, making novel entities observable and substrate-encodable.
The core pipeline (I1 entity extraction + I2 EntityDesigner + I3 scene-scoped tools) shipped in v0.7.0. Post-0.7.0 additions: archetype scaffolding — the designer infers a structural archetype (humanoid, creature, weapon…) and merges its body-part vocabulary into the LLM-generated spec, reducing hallucinated modulator names on common entity categories; manifest pre-trigger (process_manifest()) — called by the orchestrator before the first sim turn to pre-instantiate entities from the scene description, bypassing the mention threshold and arousal gate; and confirmed observe-only scene registration — imagined entities land in the EntityMap as observe-only, sensable via sense/sense_presence but with no directly callable affordance tools (the agent interacts via its own body tools).
Each stage is a gate. If any gate fails, the pipeline short-circuits gracefully — the agent falls back to verbal-only interaction with the entity.
Lightweight NLP heuristics extract entity-like noun phrases from narration text. No external model required — this runs on pure string processing.
Physical objects, creatures, weapons, environmental features, items, vehicles, NPCs
Examples: "rusty padlock", "crystal chandelier", "ancient tome", "iron golem"
Abstract concepts, body parts, clothing, emotions, time references, generic pronouns
Examples: "courage", "left arm", "leather boots", "dread", "morning"
Two strategies work in parallel:
Before imagining anything, each candidate phrase is checked against the ComponentIndex to see if an existing component already covers it:
Exact match against component names and declared synonyms from the component.synonyms YAML field.
"sword" → weapons/rusty_sword
Cosine similarity against all component signature embeddings. Threshold: 0.65. Uses the shared similarity.encoder singleton.
"old iron door" → environments/rusty_gate (0.72)
If either layer finds a match, imagination is skipped for that phrase. This prevents the system from creating duplicate components under different names.
The ComponentIndex is protected by an RLock. Multiple threads (AUT + orchestrator) can query it concurrently. Persistence uses .npy + .json sidecar — no pickle, ever.
Three gates prevent imagination from firing at inappropriate times:
Default 2 mentions before triggering. A one-off phrase ("you notice a crack in the wall") won't spawn an entity. Repeated mentions signal narrative importance.
Only fires during low-arousal idle states. Blocked when the Default Network is inhibited or recent interesting events occurred. You don't daydream while fighting.
Skipped when LLM energy is critical (<10% remaining). Falls back gracefully to verbal-only interaction with the entity.
A per-phrase lock prevents concurrent LLM calls for the same entity phrase. In multi-thread setups (AUT + orchestrator), only one thread designs a given entity; the other waits for the result. Thread-safe throughout via RLock.
When all gates pass, the ImaginationDesigner wraps the EntityDesigner and makes a single LLM call to generate a complete SEM component specification from the entity phrase and surrounding narrative context.
After generation, the spec is validated against the SEM protocol: required fields present, sensor ranges valid, modulator params typed, failure triggers well-formed. Invalid specs are discarded — the agent falls back to verbal interaction.
Imagined entities live in a separate overlay (_ephemeral_index) from the persistent component registry. This separation is architectural:
get(), has(), query()sense / sense_presence)clear_ephemeral()This means the agent learns from imagined interactions (pain avoidance, reward prediction) but with reduced confidence, reflecting the simulated origin.
All learning from imagined entities carries imagined=True provenance:
The 50% decay means the agent retains partial learning ("chandeliers can collapse") but with appropriately reduced confidence compared to verified real-world interactions. If the agent encounters the same entity type again and the interaction confirms the learned pattern, confidence rebuilds naturally through standard Rescorla-Wagner updates.
Imagined entities are registered as observe-only scene entities in the EntityMap. The agent can sense them but cannot call their affordances directly as tools. The agent interacts using its own body’s tools (the self-entity), not the scene entity’s affordances. This means:
sense and sense_presence toolsThis integrates with the scene-scoped tool system — imagined entities follow the same observe-only lifecycle as pre-authored scene entities.
A session-scoped cache prevents redundant design attempts:
Imagined entities participate in the full bio-pipeline, just like pre-authored ones:
Episodes from imagined interactions are captured with imagined=True metadata.
Causal links form from affordance outcomes. 50% confidence decay at session end.
Failure modes fire pain signals through the same cascade as real entities.
Forward models train on imagined affordance outcomes via Rescorla-Wagner.
Semantic concepts form from imagined entity interactions (modality-tagged).
Exploration directives include imagined entity affordances in the meta-prompt.
| Module | Purpose |
|---|---|
| imagination/trigger.py | Entity noun-phrase extraction, ComponentIndex lookup, design dispatch |
| imagination/designer.py | ImaginationDesigner: wraps EntityDesigner for real-time entity generation |
| imagination/cache.py | Session-scoped ImaginationCache, thread-safe, shared AUT + orchestrator |
| embodiment/component_registry.py | register_ephemeral(), clear_ephemeral(), ephemeral overlay |
| embodiment/component_index.py | Two-layer semantic discovery (alias hash + embedding cosine) |
| tools/registry.py | Scene-scoped activation, active tool cap, executor gate |
| runtime/agent_loop.py | imagination_trigger parameter on run_agentic_loop |
The imagination trigger is passed as a parameter to run_agentic_loop and fires post-state.update() on every turn. This placement means the agent has already processed the percept and updated its state before imagination considers whether to design a new entity.
The entity format imagination generates. Sensors, modulators, failure modes.
80+ pre-authored entities. ComponentIndex searches these before imagining.
Batch entity generation with gauntlet testing. Imagination is the real-time equivalent.
How imagined entity tools activate/deactivate with scene changes.