Browser Automation

CDP Chrome automation — snapshot, click, navigate, screenshot, tabs. Control Chrome through ref-based element targeting with cli-jaw browser commands.

Quick Reference

Prerequisites

• cli-jaw serve must be running — the browser skill communicates through the cli-jaw server.
• Google Chrome must be installed on the machine.
• playwright-core must be installed in the cli-jaw project (used for CDP communication).
• Optional: brew install cliclick for coordinate-based mouse operations.

Core Workflow

All browser automation follows the same loop: take a snapshot to see interactive elements, act on them by ref ID, then snapshot again to verify the result.

Use a fresh snapshot after navigation, reload, tab changes, or any action that substantially changes the page. Ref IDs belong to the latest usable snapshot and can go stale.

Quick Start

# Start an automation session (headless, no visible window)
cli-jaw browser start --agent

# Start an interactive browser (manual, visible)
cli-jaw browser start

# Navigate to a URL
cli-jaw browser navigate "https://example.com"

# Take an interactive snapshot (shows ref IDs)
cli-jaw browser snapshot --interactive

# Click a ref
cli-jaw browser click e3

# Type text and submit
cli-jaw browser type e5 "hello" --submit

# Take a screenshot
cli-jaw browser screenshot

Snapshot Output

The snapshot --interactive command returns a compact list of interactive elements on the page, each with a short ref ID you use in subsequent commands.

Ref IDs like e3 are short-lived and scoped to the latest snapshot. Always re-snapshot after page changes.

Command Reference

Support Labels

Rules and Constraints

• Ref IDs are ephemeral. They are scoped to the latest snapshot. Always re-run snapshot --interactive after navigation, reload, tab changes, or any action that changes the page.
• Prefer --interactive for snapshots to save token budget — full snapshots include non-interactive nodes and can be large.
• Use --agent for automation sessions. It starts headless and avoids popping a visible browser window.
• Never close Chrome processes found via general ps output. Only use cleanup-runtimes --close --force which validates jaw-owned processes via durable runtime records.
• Ask before reset. The reset command clears the browser profile and screenshots. Only use when the user explicitly requests it.
• evaluate is for diagnostics only. Do not pipe arbitrary user-provided JavaScript through evaluate in higher-level workflows.
• Non-DOM elements (Canvas, WebGL, cross-origin iframes, custom UI) should fall back to the vision-click skill only after confirming no usable ref exists.

Common Workflows

Web Search

cli-jaw browser start --agent
cli-jaw browser navigate "https://www.google.com"
cli-jaw browser snapshot --interactive
cli-jaw browser type e3 "search query" --submit
cli-jaw browser snapshot --interactive
cli-jaw browser click e7

Form Filling

cli-jaw browser snapshot --interactive
cli-jaw browser type e1 "John Doe"
cli-jaw browser type e2 "john@example.com"
cli-jaw browser click e3           # Submit button
cli-jaw browser snapshot           # Verify submission

Read Page Content

cli-jaw browser navigate "https://news.ycombinator.com"
cli-jaw browser text               # Plain text extraction
cli-jaw browser text --format html # HTML content
cli-jaw browser snapshot --interactive

AI Web Workflows

For ChatGPT or other web-ai workflows, use the web-ai skill. The browser skill owns primitive page control; web-ai owns structured question rendering, active-tab safety, and response baseline handling.

~해줌 Examples

Natural language requests that trigger the browser skill. Use Korean or English — both work.

Headless Mode

Three ways to run Chrome headless:

# Recommended for automation
cli-jaw browser start --agent

# Explicit headless for manual mode
cli-jaw browser start --headless

# Via environment variable
CHROME_HEADLESS=1 cli-jaw browser start

Use --agent for all agent automation. It avoids popping a visible browser window and is the expected default in automated sessions.

Environment Variables

The default CDP port is derived from the cli-jaw server port. Use cli-jaw browser start --port <port> only when you need an explicit override.

Standalone agbrowse Alternative

When you explicitly want to drive a single Chrome instance (e.g., keep one logged-in profile open, avoid running both cli-jaw serve and a second CDP session), the same browser commands are available through the standalone agbrowse CLI (npm install -g agbrowse). The flag surface is identical; only the binary prefix changes.

Runtime Cleanup

# Diagnose runtime state
cli-jaw browser doctor --json

# Dry-run: see what would be cleaned
cli-jaw browser cleanup-runtimes

# Actually close orphan jaw-owned runtimes
cli-jaw browser cleanup-runtimes --close --force

cleanup-runtimes is intentionally conservative. It only acts on a durable browser-runtime-owner.json record written by jaw-owned Chrome launches. The process command line must still match the recorded PID, CDP port, and profile. Chrome helper processes containing --type= are rejected. Never use general ps output as proof that a Chrome process is safe to close.

Recovery Strategy

When things go wrong, stop and inspect state before taking the next action.

Troubleshooting

Planned Runtime Delta

The following commands are planned parity targets from the 30_browser reference design. They are not available in the current runtime unless explicitly upgraded in a later PRD.

Planned Observe and Diagnostics

cli-jaw browser console --clear --reload --duration 3000
cli-jaw browser network --reload --duration 1000
cli-jaw browser wait 2000

Planned Actions

cli-jaw browser resize 0 0 --fullscreen
cli-jaw browser scroll down
cli-jaw browser scroll up --amount 1000

Related Skills

Notes

• Ref IDs are short-lived and should be treated as latest-snapshot scoped.
• Always re-run snapshot --interactive after navigation or major page changes.
• Prefer --interactive for token budget.
• Screenshots save to ~/.cli-jaw/screenshots/.
• start --agent should be the default for agent automation.
• Non-DOM elements such as Canvas, WebGL, cross-origin iframes, and custom UI should use the vision-click skill only as an explicit fallback.