---
name: macos-agent-ops
description: Use Homebrew-installed macos-agent to run repeatable app automation checks and routine tasks on macOS.
---

# macos-agent ops

## Contract

Prereqs:

- macOS host with Accessibility and Automation permissions granted for Terminal and target apps.
- Homebrew-installed `macos-agent` available on `PATH`.
- `agent-out` available on `PATH` for canonical project-scoped artifact directories.
- `cliclick`, `osascript`, and `im-select` available on `PATH`.
- `ABC`/`US` input source enabled in macOS Input Sources.
- For full AX surface (`ax attr/action/session/watch`), Hammerspoon runtime should be healthy (`hs` + `hs.ipc`).

Inputs:

- Command entrypoint: `$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh`.
- Optional env vars:
  - `MACOS_AGENT_OPS_INPUT_SOURCE_ID` (preferred; example: `abc`) for deterministic input source target.
  - `MACOS_AGENT_REAL_E2E_INPUT_SOURCE` (legacy fallback) if `MACOS_AGENT_OPS_INPUT_SOURCE_ID` is unset.
  - `MACOS_AGENT_OPS_SKIP_INPUT_SOURCE_SWITCH=1` to bypass automatic input source switch.
  - `CODEX_MACOS_AGENT_AX_BACKEND=auto|hammerspoon|applescript` to control AX backend preference.

Outputs:

- macos-agent JSON/text command output.
- Artifacts generated by macos-agent under a canonical `agent-out` project run directory.

Exit codes:

- `0`: success
- `1`: runtime failure or missing runtime dependency
- `2`: usage error

Failure modes:

- `wait app-active` timeouts because focus was stolen (Control Center/Spotlight/notifications).
- `window.activate` failure when app relaunch/update state is stuck; wrapper now uses `--reopen-on-fail`.
- missing `im-select` causes `input-source` failure before keyboard-input actions.
- `not authorized` / Apple Events denial from TCC permissions.
- AX target mismatch (selector too broad / stale node id) during `ax` operations.
- AX gate failures (`--gate-*`) when app/window/selector readiness conditions are not met.
- AX postcondition failures (`--postcondition-*`) when state does not change as expected after mutation.
- `observe screenshot --if-changed-baseline` path missing or unreadable.
- trace directory validation failures when using `--trace`/`--trace-dir` and path is not writable.

## Scripts (only entrypoints)

- `$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh`

## Workflow

1. Resolve Homebrew `macos-agent` binary path:

```bash
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh where
```

1. Ensure deterministic input source (default target: `abc`):

```bash
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh input-source
# or explicit source id
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh input-source --id com.apple.keylayout.ABC
```

1. Run readiness checks (`preflight` + AX list smoke check):

```bash
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh doctor
# optional target for AX smoke
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh doctor --ax-app Arc
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh doctor --ax-bundle-id com.google.Chrome
```

1. Inspect unified permission state directly (`screen_recording` / `accessibility` / `automation` / `ready`):

```bash
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh run -- \
  --format json preflight --include-probes
```

1. Run quick app foreground check with reopen recovery:

```bash
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh app-check --app Finder
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh app-check --app Arc --timeout-ms 15000
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh app-check --bundle-id com.google.Chrome
```

1. Run AX-only health check (AX tree probe):

```bash
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh ax-check --app Arc --role AXWindow
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh ax-check --app Arc --role AXTextField --title-contains Search
```

1. Use AX wait primitives before mutation to stabilize dynamic UI:

```bash
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh run -- \
  --format json wait ax-present --app Arc --role AXButton --title-contains "Compose" \
  --match-strategy contains --timeout-ms 2500 --poll-ms 80

$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh run -- \
  --format json wait ax-unique --app Arc --role AXTextField --title-contains "Search" \
  --selector-explain --timeout-ms 2500 --poll-ms 80
```

1. Run robust mutating AX actions with gating + postconditions:

```bash
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh run -- \
  --format json ax click --app Arc --role AXButton --title-contains "Play" \
  --match-strategy contains --selector-explain --reselect-before-click \
  --fallback-order ax-press,ax-confirm,frame-center,coordinate \
  --gate-app-active --gate-window-present --gate-ax-unique \
  --postcondition-focused true --postcondition-timeout-ms 2000 \
  --allow-coordinate-fallback
```

```bash
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh run -- \
  --format json ax type --app Arc --role AXTextField --title-contains "Search" \
  --text "lofi" --clear-first --submit --paste \
  --gate-app-active --gate-window-present --gate-ax-present \
  --postcondition-attribute AXValue --postcondition-attribute-value "lofi" \
  --postcondition-timeout-ms 2000 --allow-keyboard-fallback
```

1. Use selector-frame screenshots for visual proof of target alignment:

```bash
ARTIFACT_DIR="$(agent-out project --topic macos-agent-ops --mkdir)"
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh run -- \
  --format json observe screenshot --active-window \
  --role AXButton --title-contains "Play" --selector-padding 12 \
  --path "$ARTIFACT_DIR/macos-agent-selector.png"
```

1. Use diff-aware screenshots to avoid noisy artifacts when nothing changed:

```bash
ARTIFACT_DIR="$(agent-out project --topic macos-agent-ops --mkdir)"
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh run -- \
  --format json observe screenshot --active-window \
  --path "$ARTIFACT_DIR/macos-agent-state.png" \
  --if-changed --if-changed-threshold 2
```

1. Use one-shot debug bundle for triage artifacts:

```bash
ARTIFACT_DIR="$(agent-out project --topic macos-agent-ops --mkdir)"
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh run -- \
  --format json debug bundle --active-window \
  --output-dir "$ARTIFACT_DIR/macos-agent-debug"
```

1. Run routine scripted scenarios:

```bash
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh scenario --file /path/to/scenario.json
```

1. Pass through raw macos-agent commands with JSON errors + trace when needed:

```bash
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh run -- \
  --format json --error-format json --trace \
  input click --x 200 --y 160
```

## Screenshot-Based Triage Rules

- During runtime failure triage, the agent SHOULD capture an active-window screenshot before retry/remediation.
- For `wait app-active` and `window.activate` failures, screenshot capture SHOULD be first-line diagnostics.
- Screenshot artifacts MUST be written under a canonical `agent-out` run directory.
- The agent SHOULD include screenshot path + failing command + error message in the final diagnostic summary.

Recommended command (via the skill entrypoint):

```bash
ARTIFACT_DIR="$(agent-out project --topic macos-agent-failure --mkdir)"
$AGENT_HOME/skills/tools/app-runtime/macos-agent-ops/scripts/macos-agent-ops.sh run -- \
  --format json observe screenshot --active-window \
  --path "$ARTIFACT_DIR/active-window.png"
```

## Common Errors And Prevention

- `error: input source mismatch after switch attempt ...`
  - Run `.../macos-agent-ops.sh input-source --id abc` and verify `current` is `com.apple.keylayout.ABC` or `...US`.
  - Ensure `im-select` is installed: `brew install im-select`.
  - If you intentionally need non-ABC IME, set `MACOS_AGENT_OPS_SKIP_INPUT_SOURCE_SWITCH=1`.

- `error: timed out waiting for app-active ...`
  - Keep hands off keyboard/mouse while checks are running.
  - Close Control Center/Spotlight overlays and reduce notification interruptions.
  - Increase timeout (`--timeout-ms 12000` or higher) for slow app transitions.

- `error: ax.list ... timed out` during `wait ax-present` / `wait ax-unique`
  - Run `app-check` first, then retry AX waits after app focus is stable.
  - Increase AX wait timeout/poll (`--timeout-ms`, `--poll-ms`) and narrow selectors.
  - Keep backend on `auto` unless you need to force Hammerspoon behavior (`CODEX_MACOS_AGENT_AX_BACKEND`).

- `window activate failed ...`
  - Wrapper already uses `--reopen-on-fail`; rerun once to allow quit/relaunch recovery.
  - For flaky apps (e.g., Spotify updater), kill stale relauncher and retry.

- `not authorized` / Apple Events failures
  - Re-run `doctor` and fix Accessibility/Automation in System Settings.

- `selector returned zero AX matches` or ambiguous matches
  - Narrow selectors and use `--match-strategy exact|prefix|suffix|regex` when `contains` is too broad.
  - Use `--selector-explain` and `wait ax-unique` before `ax click`/`ax type`.
  - Prefer `--reselect-before-click` in dynamic pages where node ids churn.

- gate failure (`--gate-*`) before mutation
  - Add/adjust readiness gates based on context (`--gate-app-active`, `--gate-window-present`, `--gate-ax-present`, `--gate-ax-unique`).
  - Increase gate timeout/poll for slower app transitions (`--gate-timeout-ms`, `--gate-poll-ms`).

- postcondition failure (`--postcondition-*`) after mutation
  - Verify you are asserting the correct target/attribute (`AXValue`, focus state, etc.).
  - Increase postcondition timeout on animation-heavy apps (`--postcondition-timeout-ms`).

- `error: --if-changed-baseline path does not exist ...`
  - Verify baseline path exists before running capture.
  - If baseline is optional, remove `--if-changed-baseline` and let current output file be baseline.

- trace directory is not writable (`trace.write` errors)
  - Use a writable custom path with `--trace-dir`.
  - Prefer `agent-out project --topic macos-agent-ops --mkdir` for reproducible artifacts.

- Homebrew macos-agent missing
  - Install and link with Homebrew: `brew install macos-agent`.

## Practical Notes

- Prefer canonical flags in examples:
  - `--window-title-contains` (instead of legacy `--window-name`)
  - `input type --submit` (instead of legacy `--enter`)
- Keep `--error-format json` enabled in automation loops when you need machine-parseable failures.
- Use `debug bundle` first when a run fails and root cause is unclear.
- Parse preflight JSON permissions (`screen_recording` / `accessibility` / `automation` / `ready`) before mutating runs.

## References

- `skills/tools/app-runtime/macos-agent-ops/references/e2e-ref-01-preflight-focus.md`
- `skills/tools/app-runtime/macos-agent-ops/references/e2e-ref-02-finder-routine.md`
- `skills/tools/app-runtime/macos-agent-ops/references/e2e-ref-03-matrix-routine.md`
