开源计算机控制MCP

Required setup when embedding Open WebUI into your own stack

If you run Open WebUI outside the stock docker-compose.webui.yml — your own compose, Kubernetes, Portainer, or a downstream repo — there are four traps that will silently break Computer Use. All four hit us in production. Check in this order.

Step 1 — Build the image from openwebui/Dockerfile, don't pull upstream

Pulling ghcr.io/open-webui/open-webui:vX.Y.Z gives you a stock image without any of this repo's patches. Four of them are critical for UX:

Only CHAT_RESPONSE_MAX_TOOL_CALL_RETRIES keeps working on an upstream image (it's a stock Open WebUI env) — which creates a false "everything is configured" feeling.

Use build: in your downstream compose, mirroring docker-compose.webui.yml:11-15:

services:
  open-webui:
    build:
      context: ./openwebui   # path into this repo
      dockerfile: Dockerfile
      args:
        OPENWEBUI_VERSION: "0.9.2"
    image: open-webui-with-cu-patches:latest   # local tag, do not pull

Verify the patches are baked into the running container:

docker exec open-webui bash -c \
  'grep -rl "FIX_ARTIFACTS_AUTO_SHOW" /app/build/_app/immutable/chunks/ >/dev/null \
   && echo "patches applied" || echo "MISSING — you are on upstream image"'

The FIX_ARTIFACTS_AUTO_SHOW JS comment marker is injected by fix_artifacts_auto_show.py at build time as a version-stable identifier — it does not depend on minified Svelte variable names, which change with every Open WebUI release.

Step 2 — No build-arg required for preview URL detection (host-agnostic since v0.9.2.0)

fix_preview_url_detection is now fully host-agnostic. The injected JS reads the origin directly from the matched URL at runtime (_pm[1] captures the full https://host:port prefix), so the patch requires no build-time host configuration. The COMPUTER_USE_SERVER_URL build-arg has been removed from openwebui/Dockerfile.

No action needed — the patch works automatically regardless of whether you use localhost:8081, a public domain, or Docker internal DNS. The preview iframe src is always reconstructed from the URL the model wrote into the message, which in turn comes from the server's PUBLIC_BASE_URL env var.

Verify the patch is applied:

```bash docker exec open-webui bash -c \ 'grep -rl "FIX_PREVIEW_URL_DETECTION" /app/build/_app/immutable/chunks/ >/dev/null \ && echo "patches applied" || echo "MISSING — fix_preview_url_detection not baked in"'

Build your own skills — package recurring work into reusable functions

1. Start Computer Use Server (builds workspace image on first run, ~15 min)

docker compose up --build

Manual setup (if not using docker-compose)

If you run Open WebUI separately, you need to manually:

1. Go to Workspace > Tools → Create new tool → paste contents of openwebui/tools/computer_use_tools.py
2. Set Tool ID to ai_computer_use (required for filter to work)
3. Configure Valves: ORCHESTRATOR_URL = internal URL of your Computer Use Server (http://computer-use-server:8081 for Docker compose)
4. Open the tool's ⋯ → Share menu and set access to Public (grants read to both group:* and user:* wildcards) — otherwise only your admin account sees the tool and non-admin users get an empty tool list with no error
5. Go to Workspace > Functions → Create new function → paste openwebui/functions/computer_link_filter.py
6. Enable the filter: toggle Active and toggle Global in the Functions list — these are two separate switches, and active-but-not-global means the filter loads but is never applied to chats
7. In your model settings, set Function Calling = Native and Stream Chat Response = On. Or set them globally once in Admin → Settings → Models → Advanced Params (function_calling: native, stream_response: true) — that becomes DEFAULT_MODEL_PARAMS for every model.

The docker-compose stack handles all of this automatically.

2. Preview URL detection is host-agnostic (no build-arg needed since v0.9.2.0):

docker exec open-webui bash -c \ 'grep -rl "FIX_PREVIEW_URL_DETECTION" /app/build/_app/immutable/chunks/ >/dev/null \ && echo "patches applied" || echo "MISSING — fix_preview_url_detection not baked in"'

→ both must be http://computer-use-server:8081 (internal URL, Docker service DNS),

Build workspace image locally

docker build --platform linux/amd64 -t open-computer-use:latest .

Build and run full stack

docker compose up --build ```

Quick Start

```bash git clone https://github.com/Wide-Moat/open-computer-use.git cd open-computer-use cp .env.example .env

Edit .env — set OPENAI_API_KEY (or any OpenAI-compatible provider)

Model Settings (important!)

After adding a model in Open WebUI, go to Model Settings and set:

Without Function Calling: Native, the model won't invoke Computer Use tools.

Configuration

All settings via .env:

Custom Skills & Token Management (optional)

By default, all 13 built-in skills are available to everyone. For per-user skill access and custom skills, deploy the Settings Wrapper — see settings-wrapper/README.md.

Personal Access Tokens (PATs): The settings wrapper can also store encrypted per-user PATs for external services (GitLab, Confluence, Jira, etc.). The server fetches them by user email and injects into the sandbox — so each user's AI has access to their repos/docs without sharing credentials. The server-side code for token injection is implemented (docker_manager.py), but the Open WebUI tool doesn't pass the required headers yet. This is on the roadmap — if you need PAT management, open an issue.

3. Env vars reached the container:

docker exec open-webui env | grep -E 'CHAT_RESPONSE_MAX_TOOL_CALL_RETRIES|TOOL_RESULT_|ORCHESTRATOR_URL'

5. Server env (baked into system prompt AND returned to filter via header):

docker exec computer-use-server env | grep ^PUBLIC_BASE_URL=

MCP Integration

The server speaks standard MCP over Streamable HTTP. Point any MCP client at it — hosted or self-hosted.

- Hosted: https://api.yambr.com/mcp/computer_use with Authorization: Bearer <key from app.yambr.com>. Client configs and full reference live on docs.yambr.com. - Self-hosted: http://localhost:8081/mcp. Quick sanity check:

  curl -X POST http://localhost:8081/mcp \
    -H "Content-Type: application/json" \
    -H "X-Chat-Id: test" \
    -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}'
  

MCP Client Integrations

The Computer Use Server speaks standard MCP over Streamable HTTP — any MCP-compatible client can connect. Open WebUI is the primary tested frontend, but not the only option.

Open WebUI Integration

Open WebUI is an extensible, self-hosted AI interface. We use it as the primary frontend because it supports tool calling, function filters, and artifacts — everything needed for Computer Use.

Compatibility: This build is strictly built and verified against Open WebUI 0.9.2. The first 3 segments of our build version (v0.9.2.X) always match the Open WebUI base version it targets. If you run a different Open WebUI version, pick the Open Computer Use build whose first 3 version segments match yours — e.g., for Open WebUI 0.8.12 use a v0.8.12.Y build.

Why not a fork? We intentionally did not fork Open WebUI. Instead, everything is bolted on via the official plugin API (tools + functions) and build-time patches for missing features. This means you can use stock Open WebUI 0.9.2 with this build (the version that the first 3 segments of our build version v0.9.2.X match) — just install the tool and filter. Patches are applied at Docker build time; strongly recommended — 4 of them affect user-visible UX (artifacts panel, preview iframe, error banners, large tool-result handling). Pulling ghcr.io/open-webui/open-webui directly skips all of them — see Required setup when embedding Open WebUI for the full checklist.

Running Claude Code through a corporate gateway (LiteLLM, Azure, Bedrock)? See docs/claude-code-gateway.md for the three-path operator recipe.

The openwebui/ directory contains:

• tools/ — MCP client tool (thin proxy to Computer Use Server). Required — this is the bridge between Open WebUI and the sandbox.
• functions/ — System prompt injector + file link rewriter + archive button. Required — without it the model doesn't know about skills and file URLs.
• patches/ — Build-time fixes for artifacts, error handling, file preview. Optional but recommended — improves UX significantly.
• init.sh — Auto-installs tool + filter on first startup. Optional — you can install manually via Workspace UI instead.
• Dockerfile — Builds a patched Open WebUI image with auto-init. Optional — use stock Open WebUI + manual setup if you prefer.