Official Docs / AI Chat & Interaction

Active Agent State

C++ Subsystem Direct Connection

Chapter 2: How to Chat (4 Built-in Agent Modes)

Inside the built-in UMG MCP chat panel on the right side of the Unreal Editor, AI communication is not a static one-size-fits-all session. Based on different development phases and contexts, this plugin categorizes AI chat states into four precise Agent modes: chat, develop, learning, and task at the C++ subsystem layer.

1. The 4 Agent Operational Logic & C++ Mechanisms

Chat Mode (Default Session Agent - FUmgMcpDefaultChatAgent)

Chat

Targeting: General context-aware conversations. Ideal for basic technical inquiries, design brainstorming, and conceptual clarification.

C++ Mechanism: At the start of each query cycle, it triggers RestoreConversationContext(), pulling the complete substance of the previous conversation (historical JSON message snap) from local persistent session files. This historical sequence serves as the message context prefix submitted to the LLM provider, guaranteeing zero context fragmentation.

Develop Mode (Dynamic Development Agent - FUmgMcpDevelopAgent)

Develop

Targeting: Highly context-sensitive UI authoring and widget editing. Focuses on high-precision UI layout adjustments and programmatic widget generation.

C++ Mechanism: Upon interaction, it calls the unique DynamicallyUpdatePromptAndTools() method. It inspects the currently active UMG asset being edited in the Unreal Editor, dynamically re-synthesizes the session's System Instruction (system prompts), and filters permitted MCP tools based on the currently selected widget types. For instance, Canvas Slot adjustment tools are only exposed when CanvasPanel itself is selected, greatly narrowing the AI generation scope and boosting tool execution success rates.

Learning Mode (Read-Only Perceptual Agent - FUmgMcpLearningAgent)

Learning

Targeting: Deep structural parsing, layout hierarchy perception, and step-by-step guidance on complex, pre-existing UMG widget structures.

C++ Mechanism: The plugin loads the built-in learning.json tool configurations silently. During query evaluation, it executes ResolveTargetAndTreeContext() designed specifically for data optimization: it computes and compares the hash value of the current active viewport widget hierarchy. **Only** when you physically alter the layout structure does it bundle and upload the new tree representation to the context. Otherwise, it safely recycles the read-only cache, completely eliminating repetitive structural queries and saving huge token counts/network latency.

Task Mode (Task Handover Agent - FUmgMcpTaskAgent)

Task

Targeting: Executing self-contained development tasks with strong consistency and comprehensive transactional safety boundaries.

C++ Mechanism: The task handover flow runs under rigorous engineering controls. When the AI receives a special task_begin tool call, it fires AcceptTask(), saving your current editor focus path (e.g. `PreviousWidget`) into a private backup variable CapturedPreviousTarget. It then locks the editor viewport strictly onto the target task asset, masking any other asset modifications. Once the completion trigger task_end is issued, TaskEnd() executes, unlocking editing states and returning focus to the original active asset without any distortion.

🎨 Slate Chat UI Approval Queue & direct Memory execution

Whenever an active Agent attempts an action that mutates your UMG assets (e.g., "create a button widget" or "delete a slot layout"), the plugin's underlying SUmgMcpMcpApprovalQueue (Slate Approval UI) interceptor activates. It presents an interactive, highlighted authorization card inside your chat timeline.

💡 Operational Step-by-Step Experience:
  1. Submit Query: You input a prompt, e.g. "Create a button widget named WinyunqButton under the Canvas panel".
  2. AI Plan & Action Formulation: The Agent processes the request and silently queues the tool execution metadata card (such as create_widget) onto the Slate Chat view.
  3. Approval Prompts: The interactive bubble displays a blue Accept and a gray Reject trigger button.
  4. Zero Network Proxy Direct Connection: Clicking Accept routes the raw instruction directly to FFabServerHttpClient::ExecuteTool. It is instantly executed on Unreal's Game Thread in memory, refreshing the Slate Designer View and producing a real-time UMG object. There is no network loopback or roundtrip proxy!

📸 Screenshot Placeholder

[IMAGE_PLACEHOLDER: Unreal UMG MCP Built-in Chat Panel Window Layout]

Please take a screenshot of the UMG MCP chat window docked in the Unreal Editor, displaying the bottom Selector containing the chat/develop/learning/task modes.