# Skills & Marketplace

Skills are files that describe multi-step iPhone automation flows as intents, not scripts. They can be written in **SKILL.md** format (recommended) or **YAML** format (legacy). The MCP server provides the tools; skills teach the AI what to do with them.

## Overview

The system has two layers:

1. **This repository** (`mirroir-mcp`) — provides 32 MCP tools for iPhone interaction
2. **Skill repositories** (e.g., [jfarcand/mirroir-skills](https://github.com/jfarcand/mirroir-skills)) — provide reusable skill files (SKILL.md or YAML) + plugin discovery

Skills are intentionally simple. Steps like `tap: "Email"` don't specify pixel coordinates — the AI uses `describe_screen` for fuzzy OCR matching and adapts to unexpected dialogs, layout changes, and timing differences.

## Plugin Discovery

Skills can be installed via AI coding assistant plugin systems or manually.

### Claude Code

```bash
claude plugin marketplace add jfarcand/mirroir-skills
claude plugin install skills@mirroir-skills
```

Plugin metadata lives in `.claude-plugin/marketplace.json` in the skill repository. The `SKILL.md` file in the skill repo teaches Claude how to interpret skill steps.

### GitHub Copilot CLI

```bash
copilot plugin marketplace add jfarcand/mirroir-skills
copilot plugin install skills@mirroir-skills
```

Plugin metadata lives in `.github/plugin/marketplace.json` in the skill repository.

### Manual Installation

Clone or copy skill files (`.md` or `.yaml`) into one of the scan directories:

```bash
# Global — available in all projects
git clone https://github.com/jfarcand/mirroir-skills.git \
    ~/.mirroir-mcp/skills/

# Project-local — available only in current project
mkdir -p .mirroir-mcp/skills/
cp my-skill.md .mirroir-mcp/skills/
```

Project-local skills with the same filename override global ones. When both a `.md` and `.yaml` file exist with the same stem name, the `.md` file takes precedence.

## Skill Format

Skills can be written in SKILL.md (recommended) or YAML (legacy). Both formats support the same fields and step types.

### SKILL.md Format (Recommended)

SKILL.md uses YAML front matter for metadata and a markdown body with natural-language steps:

```markdown
---
version: 1
name: Send Slack Message
app: Slack
tags: ["messaging", "slack"]
---

Send a direct message to a contact in Slack.

## Steps

1. Launch **Slack**
2. Wait for "Home" to appear
3. Tap "Direct Messages"
4. Wait for "${RECIPIENT}" to appear
5. Tap "${RECIPIENT}"
6. Wait for "Message" to appear
7. Tap "Message"
8. Type "${MESSAGE:-Hey, just checking in!}"
9. Press **Return**
10. Screenshot: "message_sent"
```

Convert existing YAML skills with `mirroir migrate`:

```bash
mirroir migrate skill.yaml                            # convert a single file
mirroir migrate --dir path/to/skills/                 # convert all YAML files in a directory
mirroir migrate --output-dir ./converted/ skill.yaml  # write .md files to alternate directory
mirroir migrate --dry-run skill.yaml                  # preview without writing
```

### YAML Format (Legacy)

### Required Fields

| Field | Type | Description |
|-------|------|-------------|
| `name` | string | Human-readable skill name |
| `app` | string | Target app(s), comma-separated for cross-app flows |
| `description` | string | What the skill does |
| `steps` | list | Ordered list of step objects |

### Optional Metadata

| Field | Type | Description |
|-------|------|-------------|
| `version` | integer | Skill format version (default: 1). Used for future format evolution. |
| `ios_min` | string | Minimum iOS version (e.g., `"17.0"`) |
| `locale` | string | Expected device locale (e.g., `"fr-CA"`) |
| `tags` | list | Categorization tags (e.g., `["productivity", "messaging"]`) |

### Example

```yaml
name: Send Slack Message
app: Slack
description: Send a direct message to a contact in Slack

steps:
  - launch: "Slack"
  - wait_for: "Home"
  - tap: "Direct Messages"
  - wait_for: "${RECIPIENT}"
  - tap: "${RECIPIENT}"
  - wait_for: "Message"
  - tap: "Message"
  - type: "${MESSAGE:-Hey, just checking in!}"
  - press_key: "return"
  - screenshot: "message_sent"
```

## Step Types

Each step is a single key-value pair in the `steps` list. The AI interprets each step using the appropriate MCP tool.

| Step Type | Value | MCP Tool Used | Description |
|-----------|-------|---------------|-------------|
| `launch` | App name (string) | `launch_app` | Open an app via Spotlight search |
| `tap` | Element text (string) | `describe_screen` + `tap` | Find element by OCR text match, tap its coordinates |
| `type` | Text to type (string) | `type_text` | Type text into the currently focused field |
| `press_key` | Key name (string) | `press_key` | Press a special key (return, escape, tab, etc.) with optional modifiers |
| `swipe` | Direction (string) | `swipe` | Swipe in a direction (up, down, left, right) |
| `wait_for` | Element text (string) | `describe_screen` (poll) | Wait until text appears on screen (retry with describe_screen) |
| `assert_visible` | Element text (string) | `describe_screen` | Verify text is visible; fail the skill if not found |
| `assert_not_visible` | Element text (string) | `describe_screen` | Verify text is NOT visible; fail the skill if found |
| `screenshot` | Label (string) | `screenshot` | Capture a screenshot with a descriptive label |
| `shake` | `true` | `shake` | Trigger a shake gesture (debug menus, undo) |
| `press_home` | `true` | `press_home` | Return to the home screen |
| `open_url` | URL (string) | `open_url` | Open a URL in Safari |
| `scroll_to` | Element text (string) | `scroll_to` | Scroll until a text element becomes visible via OCR |
| `reset_app` | App name (string) | `reset_app` | Force-quit an app via the App Switcher |
| `set_network` | Mode (string) | `set_network` | Toggle network settings (airplane_on/off, wifi_on/off, cellular_on/off) |
| `measure` | Object with `action`, `until`, `max`? | `measure` | Time a screen transition after an action |
| `target` | Target name (string) | `switch_target` | Switch to a different automation target window |
| `remember` | Instruction (string) | _(AI-interpreted)_ | Tell the AI to extract and remember data from the current screen |
| `long_press` | Element text (string) | `describe_screen` + `long_press` | Find element by OCR, long press on it |
| `drag` | Object with `from`/`to` | `describe_screen` + `drag` | Find elements by OCR, drag between them |
| `condition` | Object with `if_visible`/`if_not_visible` + `then`/`else` | `describe_screen` | Branch based on screen state — see below |
| `repeat` | Object with loop mode + `max` + `steps` | `describe_screen` (poll) | Loop over steps until a screen condition is met — see below |

### Conditions

Skills can branch using `condition` steps. The AI calls `describe_screen` to evaluate the condition, then executes the matching branch:

```yaml
- condition:
    if_visible: "Unread"        # or if_not_visible
    then:
      - tap: "Unread"
      - tap: "Archive"
    else:                       # optional
      - screenshot: "empty"
```

If the condition is true, the `then` steps execute. If false and `else` is present, those steps execute instead. Steps inside branches are regular steps, including nested conditions (up to 3 levels deep).

### Repeats

Skills can loop using `repeat` steps. The AI checks a screen condition before each iteration and stops when the condition fails or `max` is reached:

```yaml
- repeat:
    while_visible: "Unread"     # or until_visible, or times: N
    max: 10                      # required safety bound
    steps:
      - tap: "Unread"
      - tap: "Archive"
      - tap: "< Back"
```

Loop modes: `while_visible: "Label"` (continue while present), `until_visible: "Label"` (continue until appears), `times: N` (fixed count). The `max` field is always required to prevent infinite loops. Steps inside are regular steps, including conditions and nested repeats (up to 3 levels deep).

## Variable Substitution

### Environment Variables: `${VAR}`

Variables wrapped in `${...}` are resolved from environment variables at `get_skill` time (before the AI sees the skill content). This works for both SKILL.md and YAML formats.

```yaml
- type: "${TEST_EMAIL}"           # Required — left as ${TEST_EMAIL} if unset
- type: "${CITY:-Montreal}"       # Optional — defaults to "Montreal" if unset
```

| Syntax | Behavior |
|--------|----------|
| `${VAR}` | Substituted with env var value. Left as-is if unset (AI flags it as missing). |
| `${VAR:-default}` | Substituted with env var value. Uses `default` if env var is unset. |

### AI-Remembered Data: `{var}`

Single-brace variables are placeholders for data the AI extracts during skill execution via `remember:` steps:

```yaml
- remember: "Read the commute time and ETA from the navigation screen."
# AI extracts: commute_time = "35 min", eta = "9:15 AM"

- type: "On my way! {commute_time} to the office (ETA {eta})"
# AI substitutes: "On my way! 35 min to the office (ETA 9:15 AM)"
```

These are never resolved by the server — they exist in the YAML for the AI to interpret contextually.

## AI Execution Model

Skills are executed by AI, not by a deterministic runner. This is by design:

**Why AI instead of a script runner:**
- **Fuzzy matching:** `tap: "Email"` works even if the actual text is "Email Address" or "E-mail"
- **Scroll discovery:** If an element isn't visible, the AI knows to scroll down and re-scan
- **Dialog dismissal:** Unexpected permission dialogs, update prompts, or alerts are handled naturally
- **Retry logic:** Network timeouts, slow animations, and loading states are handled contextually
- **Layout adaptation:** Different iPhone models, font sizes, and orientations are handled without pixel-perfect coordinates

**The remember/recall pattern:**
1. A `remember:` step tells the AI to extract specific data from the current screen
2. The AI uses `describe_screen` to read the screen content
3. The AI stores the extracted values internally
4. Later `{variable}` references in `type:` steps use those stored values

This enables cross-app data flows — read data from one app, switch apps, then use that data in another.

## Skill Validation

Skill files should be validated for:

- **Required fields:** `name`, `app`, `description`, `steps` must all be present
- **Step types:** Each step key must be a recognized step type
- **Variable syntax:** `${VAR}` patterns must have valid identifier names
- **Non-empty steps:** The `steps` list must contain at least one step

## Available Skills

Skills are maintained in the [mirroir-skills](https://github.com/jfarcand/mirroir-skills) repository. Install them via the plugin system (see [Plugin Discovery](#plugin-discovery) above) or clone manually into `~/.mirroir-mcp/skills/`.

The mirroir-skills repository is automatically tagged and released with each mirroir-mcp release. Plugin marketplace installations always get a compatible version. If you cloned manually, run `git pull` after upgrading mirroir-mcp to stay in sync.

## Contributing a Skill

Checklist for adding a new skill:

1. **Create the skill file** (`.md` for SKILL.md format, or `.yaml` for legacy) in the appropriate directory under `skills/`
   - `apps/<app-name>/` for app-specific skills
   - `testing/<framework>/` for test automation skills

2. **Include all required fields:** `name`, `app`, `description`, `steps`

3. **Use environment variables** for user-specific data (`${RECIPIENT}`, `${MESSAGE}`)

4. **Provide defaults** for optional parameters (`${CITY:-Montreal}`)

5. **Add verification steps:**
   - `wait_for:` after navigation to confirm the expected screen loaded
   - `assert_visible:` to verify the action succeeded
   - `screenshot:` at key points for visual confirmation

6. **Test the skill** end-to-end on a real device with `get_skill` + manual execution

7. **Keep steps intent-based** — describe *what* to do, not *how* to do it. Let the AI handle coordinates, scrolling, and timing.
