# AI Agent Integration Guide (Advanced)

> **Looking for the easiest setup?** See [AI Agent Quick Start](ai-agent-setup.md) which uses `muster standalone` for single-command integration.

## What we're doing here

This **advanced guide** walks you through setting up Muster with separate server and agent processes for production environments. This approach provides visible logs, supports multiple MCP clients, and is ideal for complex deployments.

**Use this guide when you need:**

- Visible server logs for debugging
- Multiple IDE/MCP clients connecting to one server
- Production deployment patterns
- Environment-specific configurations

**For simple IDE integration, use `muster standalone` instead.**

## Understanding Muster's Architecture

**Critical Concept**: Before setting up AI integration, understand Muster's meta-tools interface:

### Aggregator Server (`muster serve`)

- **Exposes**: 11 meta-tools (`list_tools`, `call_tool`, etc.) as the only interface
- **Contains**: 36 core tools + external MCP tools (accessed via `call_tool`)
- **Purpose**: Business logic, tool execution, service management

### Agent (`muster agent --mcp-server`)

- **Role**: Transport bridge (stdio ↔ HTTP/SSE) + OAuth handling
- **Purpose**: Enable non-OAuth MCP clients to connect to authenticated servers
- **Note**: OAuth-capable clients can connect directly to the server

**Your AI agent uses meta-tools from the server to access all functionality.**

## Before we start

### What you'll need

- **OS**: Linux, macOS, or Windows
- **Go**: Version 1.21 or later
- **IDE**: VSCode, Cursor, or another MCP-compatible environment
- **Network**: Ability to run local processes

### What you should know

- Basic JSON configuration
- How to configure extensions in your IDE
- Basic command-line usage
- Understanding of Muster's two-layer architecture

## Getting Muster installed

### Build from source

```bash
# Clone the repo
git clone https://github.com/giantswarm/muster.git
cd muster

# Build and install
go install

# Verify installation
muster version
```

## Setting things up

### Step 1: Configure and Start Muster Server

Muster uses a configuration directory at `.muster/` by default:

```bash
# Create the config directory (optional - created automatically)
mkdir -p .muster/{mcpservers,workflows}

# Start Muster server (keep this running)
muster serve
```

**Note**: Keep this terminal open - the server needs to stay running for the agent to connect.

### Step 2: Add MCP servers (optional)

You can add MCP servers to extend muster's capabilities:

```yaml
# .muster/mcpservers/filesystem.yaml
apiVersion: muster.giantswarm.io/v1alpha1
kind: MCPServer
metadata:
  name: filesystem-tools
  namespace: default
spec:
  description: "File system operations for development"
  toolPrefix: "fs"
  type: stdio
  autoStart: true
  command: ["npx", "@modelcontextprotocol/server-filesystem", "/workspace"]
  env:
    DEBUG: "1"
```

### Step 3: Test Muster

```bash
# Test interactive mode
muster agent --repl

# In the REPL, try meta-tools:
list tools                                         # Discover all available tools
call core_config_get {}     # Execute core tools
call core_service_list {}   # Execute service tools
call core_mcpserver_list {} # Execute MCP server tools
```

## Connecting your IDE

### VSCode setup

**Configure MCP extension**
Add this to your VSCode settings:

```json
{
  "mcpServers": {
    "muster": {
      "command": "muster",
      "args": ["agent", "--mcp-server"]
    }
  }
}
```

### Cursor setup

**Add Muster as MCP server**

```json
{
  "mcpServers": {
    "muster": {
      "command": "muster",
      "args": ["agent", "--mcp-server"]
    }
  }
}
```

### Other MCP clients

For other tools that support MCP, use the same configuration pattern:

```json
{
  "mcpServers": {
    "muster": {
      "command": "muster",
      "args": ["agent", "--mcp-server"],
      "description": "Muster unified tool interface"
    }
  }
}
```

## What your AI agent can do

Once configured, your AI agent will have access to **11 meta-tools** from the server that provide access to all Muster functionality:

### Server Meta-Tools (What AI Agents See)

**Tool Discovery:**

- `list_tools` - Discover all available tools
- `describe_tool` - Get detailed information about any tool
- `filter_tools` - Filter tools by name patterns or descriptions
- `list_core_tools` - List built-in Muster tools specifically

**Tool Execution:**

- `call_tool` - Execute any tool by name

**Resource Access:**

- `list_resources` - List available resources
- `get_resource` - Retrieve resource content
- `describe_resource` - Get resource details

**Prompt Access:**

- `list_prompts` - List available prompts
- `get_prompt` - Execute prompt templates
- `describe_prompt` - Get prompt details

### Actual Tools (Accessed via `call_tool`)

When your AI agent uses `call_tool`, it can execute any of the **36+ aggregator tools**:

**Configuration Management (5 tools):**

- `core_config_get` - Get complete system configuration
- `core_config_get_aggregator` - Get aggregator-specific settings
- `core_config_update_aggregator` - Modify aggregator configuration
- `core_config_save` - Persist configuration changes
- `core_config_reload` - Reload from configuration files

**Service Management:**

- `core_service_list` - List all services
- `core_service_start/stop/restart` - Control service lifecycle
- `core_service_status` - Monitor service health

**Workflow Orchestration (9 tools):**

- `core_workflow_list` - List available workflows
- `core_workflow_create` - Define multi-step processes
- `core_workflow_get` - Get workflow details
- `core_workflow_available` - Check workflow dependencies
- `core_workflow_update/delete` - Modify or remove workflows
- `core_workflow_validate` - Validate workflow configurations
- `core_workflow_execution_list` - View execution history
- `core_workflow_execution_get` - Get execution details
- `workflow_<name>` - Execute specific workflows (auto-generated)

**MCP Server Management (6 tools):**

- `core_mcpserver_list` - List external tool providers
- `core_mcpserver_create` - Add new MCP servers
- `core_mcpserver_get` - Get server details
- `core_mcpserver_update/delete` - Modify or remove servers
- `core_mcpserver_validate` - Validate server configurations

**External Tools (Variable):**

- Tools from configured MCP servers (e.g., `x_kubernetes_*`, `x_prometheus_*`)

## Example AI interactions

Your AI agent can now help you with infrastructure tasks using the correct two-layer pattern:

**"Show me the current system configuration"**

```bash
AI executes: call core_config_get {}
```

**"List all running services"**

```bash
AI executes: call core_service_list {}
```

**"Execute the authentication workflow"**

```bash
AI executes: call workflow_auth-workflow {
  "cluster": "my-cluster",
  "profile": "default"
})
```

**"Check workflow execution history"**

```bash
AI executes: call core_workflow_execution_list {
  "workflow_name": "auth-workflow"
}
```

**"Discover available tools"**

```bash
AI executes: list tools
```

**"Filter tools for service management"**

```bash
AI executes: filter tools workflow_*
```

## OAuth Authentication

When connecting to a remote Muster Server that requires authentication, the agent handles OAuth 2.1 automatically.

### How OAuth Authentication Works

1. **Connection Attempt**: The agent tries to connect to the Muster Server
2. **401 Detection**: If the server requires authentication, it returns a 401 with OAuth metadata
3. **Pending Auth State**: The agent exposes a synthetic `authenticate_muster` tool
4. **User Authentication**: Call `authenticate_muster` to get an auth URL, then sign in via browser
5. **Automatic Upgrade**: After sign-in, the agent connects and exposes all real tools
6. **SSO Chain**: The agent automatically authenticates with remote MCP servers (disable with `--disable-auto-sso`)

### Connecting to a Protected Remote Server

```json
{
  "mcpServers": {
    "muster": {
      "command": "muster",
      "args": ["agent", "--mcp-server", "--endpoint=https://muster.example.com/mcp"]
    }
  }
}
```

When you first connect:
1. Your AI assistant will see the `authenticate_muster` tool
2. Call it to receive an authentication URL
3. Open the URL in your browser and sign in
4. The agent automatically reconnects and exposes all tools

### Automatic SSO for Remote MCP Servers

By default, the agent automatically authenticates with remote MCP servers after you sign in to the main Muster Server. This leverages Single Sign-On (SSO) since remote servers typically share the same Identity Provider (Dex).

**What happens (default behavior):**
1. You sign in once via `authenticate_muster`
2. The agent detects remote servers needing auth (e.g., `authenticate_github`, `authenticate_gitlab`)
3. Browser tabs open automatically for each remote server
4. SSO redirects complete authentication without additional login prompts
5. A summary shows how many servers were authenticated

**To disable automatic SSO:**

```json
{
  "mcpServers": {
    "muster": {
      "command": "muster",
      "args": ["agent", "--mcp-server", "--disable-auto-sso", "--endpoint=https://muster.example.com/mcp"]
    }
  }
}
```

**With `--disable-auto-sso`:** Remote servers remain unauthenticated until you explicitly call their `authenticate_*` tools. This gives you full control over when browser tabs open.

**Note:** The agent opens browser tabs with a 2-second delay between each to allow SSO redirects to complete. If browser pop-ups are blocked, the agent will display the URLs for manual authentication.

### Token Persistence and Security

OAuth tokens are stored securely on your local filesystem:

- **Location**: `~/.config/muster/tokens/`
- **File permissions**: `0600` (owner read/write only)
- **Directory permissions**: `0700` (owner access only)
- **Format**: JSON files with hashed server URL as filename

Subsequent connections reuse existing valid tokens, so you only need to authenticate once per server until the token expires.

**Security Considerations**:
- Tokens contain sensitive credentials - treat this directory like SSH keys
- To revoke access: `rm -rf ~/.config/muster/tokens/`
- For shared machines: ensure proper user isolation
- Tokens have a 60-second expiry buffer to handle network latency

### Client Registration (CIMD)

The Muster Agent uses a Client ID Metadata Document (CIMD) hosted on GitHub Pages for OAuth client identification:

- **Client ID**: `https://giantswarm.github.io/muster/muster-agent.json`
- **Redirect URI**: `http://localhost:3000/callback`

For self-hosted Muster deployments, ensure your IdP trusts this client ID.

## Troubleshooting

### Connection issues

- Ensure `muster serve` is running
- Check that the agent can connect: `muster agent --repl`
- Verify IDE MCP extension is properly installed

### Tool not available

- Check if MCP servers are running: Ask agent to run `call core_mcpserver_list {})`
- Verify tool availability by testing meta-tools: Ask agent to run `list_tools()`
- Check aggregator status: Ask agent to run `call core_service_status {"name": "mcp-aggregator"})`

### Configuration problems

- Check configuration directory: `.muster/`
- Verify YAML syntax in configuration files
- Check muster logs for error messages
- Test configuration: Ask agent to run `call core_config_get {})`

### OAuth authentication issues

- **Token expired**: Clear tokens with `rm -rf ~/.config/muster/tokens/` and re-authenticate
- **Callback not received**: Ensure port 3000 is available and no firewall blocks localhost
- **Client not registered**: For self-hosted servers, verify the CIMD client ID is trusted by your IdP
- **Wrong issuer**: Check that the server's WWW-Authenticate header points to your IdP

### SSO chain issues

- **Browser pop-ups blocked**: Some browsers block automatic tab opening. The agent will display URLs for manual authentication if this happens
- **SSO not working**: Ensure all remote MCP servers use the same IdP (Dex) as the Muster Server
- **Too many tabs opening**: This is expected behavior. Use `--disable-auto-sso` if you prefer manual control
- **Partial authentication**: Check the agent logs for which servers failed. You can manually call their `authenticate_*` tools

## Next steps

1. **Explore meta-tools**: Use `muster agent --repl` to try all 11 server meta-tools
2. **Build workflows**: Automate processes in `.muster/workflows/`
3. **Add MCP servers**: Integrate external tools in `.muster/mcpservers/`

### Real Examples from Current Configuration

Based on your `.muster` setup, you can try:

- **Workflows**: `auth-workflow`, `login-workload-cluster`, `connect-monitoring`
- **MCP Servers**: `kubernetes`, `prometheus`, `grafana`

### Key Usage Patterns for AI Agents

Remember these patterns when working with AI agents:

```bash
# ✅ Correct: How AI agents work with Muster (via server meta-tools)
list_tools()                                        # Discover tools
call_tool(name="core_service_list", arguments={})   # Execute tools
filter_tools(pattern="workflow_*")                  # Filter tools

# ❌ Wrong: These tools don't exist at the MCP interface level
core_service_list()                                 # Must use call_tool
workflow_auth-workflow()                            # Must use call_tool
```

This architecture enables:

- **Unified access** to all tool types (core, workflow, external)
- **Dynamic discovery** of available capabilities
- **Session-scoped visibility** based on authentication state
- **OAuth-capable clients** can connect directly to the server

For comprehensive examples, see the test scenarios in `internal/testing/scenarios/`.
