开源计算机控制MCP

Required setup when embedding Open WebUI into your own stack

If you run Open WebUI outside the stock docker-compose.webui.yml — your own compose, Kubernetes, Portainer, or a downstream repo — there are four traps that will silently break Computer Use. All four hit us in production. Check in this order.

Step 1 — A stock upstream image is all this repo needs

Install the tool and the filter and Computer Use works against ghcr.io/open-webui/open-webui as published. Nothing here has to be rebuilt.

This used to be the opposite: the repository carried eight patch scripts and a Dockerfile that applied them to an already-built image, and pulling upstream silently skipped all of them. Those patches are gone. Three of the problems they addressed have since been fixed upstream; the rest live as source commits in a fork, which is a separate concern from running this integration.

Preview URL detection needs no build-time host configuration either — the iframe origin is read from the URL the model wrote, which comes from the server's PUBLIC_BASE_URL.

Step 3 — Two URL settings, two roles (public vs internal)

v4.0.0: the old "three FILE_SERVER_URL places that must match" footgun is gone. There are now only two places and two distinct roles — public (browser-reachable) vs internal (Docker-local). The COMPUTER_USE_SERVER_URL build-arg was removed in v0.9.2.0 — fix_preview_url_detection is now host-agnostic (see Step 2).

⚠️ Do NOT point ORCHESTRATOR_URL at your public domain. It technically works, but every MCP request then goes browser→CDN→Traefik→container. Any hiccup in that chain kills the stream mid-tool-call and the user sees MCP call failed: Session terminated. Stay inside the Docker network.

The filter no longer has a public-URL Valve at all — it reads the public URL from the server's X-Public-Base-URL response header and caches it alongside the prompt. One public knob, one internal knob.

See also docs/openwebui-filter.md.

Step 4 — Four env vars on the open-webui container

Copy-paste into your downstream compose environment: block:

services:
  open-webui:
    environment:
      # --- Computer Use required env vars (read by build-time patches) ---
      - CHAT_RESPONSE_MAX_TOOL_CALL_ITERATIONS=200
      - TOOL_RESULT_MAX_CHARS=50000
      - TOOL_RESULT_PREVIEW_CHARS=2000
      # Internal URL of the Computer Use server — seeded by init.sh into both
      # Tool and Filter Valves, and read by the fix_large_tool_results patch.
      # Same Docker network: use the service DNS name.
      - ORCHESTRATOR_URL=http://computer-use-server:8081

Note: the last three are no-ops if the image is upstream ghcr.io — they need fix_large_tool_results from Step 1.

Step 5 — Filter must be global, tool must be public-read

Open WebUI has two separate switches for each function (is_active and is_global) and two required grants for each tool (group:* + user:*). The stock init.sh does this for you; manual / custom deployments commonly miss one side and then spend hours wondering why "everything is installed but nothing works."

Verify against the database (Postgres used by the stock stack; see docker-compose.webui.yml:53):

```bash

Build your own skills — package recurring work into reusable functions

1. Start Computer Use Server (builds workspace image on first run, ~15 min)

docker compose up --build

Manual setup (if not using docker-compose)

If you run Open WebUI separately, you need to manually:

1. Go to Workspace > Tools → Create new tool → paste contents of openwebui/tools/computer_use_tools.py
2. Set Tool ID to ai_computer_use (required for filter to work)
3. Configure Valves: ORCHESTRATOR_URL = internal URL of your Computer Use Server (http://computer-use-server:8081 for Docker compose)
4. Open the tool's ⋯ → Share menu and set access to Public (grants read to both group:* and user:* wildcards) — otherwise only your admin account sees the tool and non-admin users get an empty tool list with no error
5. Go to Workspace > Functions → Create new function → paste openwebui/functions/computer_link_filter.py
6. Enable the filter: toggle Active and toggle Global in the Functions list — these are two separate switches, and active-but-not-global means the filter loads but is never applied to chats
7. In your model settings, set Function Calling = Native and Stream Chat Response = On. Or set them globally once in Admin → Settings → Models → Advanced Params (function_calling: native, stream_response: true) — that becomes DEFAULT_MODEL_PARAMS for every model.

The docker-compose stack handles all of this automatically.

→ both must be http://computer-use-server:8081 (internal URL, Docker service DNS),

Build workspace image locally

docker build --platform linux/amd64 -t open-computer-use:latest .

Build and run full stack

docker compose up --build ```

Quick Start

```bash git clone https://github.com/Wide-Moat/open-computer-use.git cd open-computer-use cp .env.example .env

Edit .env — set OPENAI_API_KEY (or any OpenAI-compatible provider)

Model Settings (important!)

After adding a model in Open WebUI, go to Model Settings and set:

Without Function Calling: Native, the model won't invoke Computer Use tools.

Configuration

All settings via .env:

Custom Skills & Token Management (optional)

By default, all 13 built-in skills are available to everyone. For per-user skill access and custom skills, deploy the Settings Wrapper — see settings-wrapper/README.md.

Personal Access Tokens (PATs): The settings wrapper can also store encrypted per-user PATs for external services (GitLab, Confluence, Jira, etc.). The server fetches them by user email and injects into the sandbox — so each user's AI has access to their repos/docs without sharing credentials. The server-side code for token injection is implemented (docker_manager.py), but the Open WebUI tool doesn't pass the required headers yet. This is on the roadmap — if you need PAT management, open an issue.

1. Env vars reached the container:

docker exec open-webui env | grep -E 'CHAT_RESPONSE_MAX_TOOL_CALL_ITERATIONS|TOOL_RESULT_|ORCHESTRATOR_URL'

5. Server env (baked into system prompt AND returned to filter via header):

docker exec computer-use-server env | grep ^PUBLIC_BASE_URL=

MCP Integration

The server speaks standard MCP over Streamable HTTP. Point any MCP client at your own deployment.

- Self-hosted: http://localhost:8081/mcp. Quick sanity check:

  curl -X POST http://localhost:8081/mcp \
    -H "Content-Type: application/json" \
    -H "X-Chat-Id: test" \
    -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}'
  

MCP Client Integrations

The Computer Use Server speaks standard MCP over Streamable HTTP — any MCP-compatible client can connect. Open WebUI is the primary tested frontend, but not the only option.

Open WebUI Integration

Open WebUI is an extensible, self-hosted AI interface. We use it as the primary frontend because it supports tool calling, function filters, and artifacts — everything needed for Computer Use.

Compatibility: This build is strictly built and verified against Open WebUI 0.11.0. The first 3 segments of our build version (v0.11.0.X) always match the Open WebUI base version it targets. If you run a different Open WebUI version, pick the Open Computer Use build whose first 3 version segments match yours — e.g., for Open WebUI 0.8.12 use a v0.8.12.Y build.

Why not a fork? Computer Use itself is not a fork: it bolts on through the official plugin API — tools and functions — so stock Open WebUI works with just the tool and filter installed. (A separate fork does exist for changes that cannot be expressed as plugins, but nothing in this repository depends on it.)

Running Claude Code through a corporate gateway (LiteLLM, Azure, Bedrock)? See docs/claude-code-gateway.md for the three-path operator recipe.

The openwebui/ directory contains:

• tools/ — MCP client tool (thin proxy to Computer Use Server). Required — this is the bridge between Open WebUI and the sandbox.
• functions/ — System prompt injector + file link rewriter + archive button. Required — without it the model doesn't know about skills and file URLs.
• patches/ — Build-time fixes for artifacts, error handling, file preview. Optional but recommended — improves UX significantly.
• init.sh — Auto-installs tool + filter on first startup. Optional — you can install manually via Workspace UI instead.
• init.sh — Installs the tool and filter on first start and sets their valves.