gpustack Agent工作流

Prerequisites

1. A node with at least one NVIDIA GPU. For other GPU types, please check the guidelines in the GPUStack UI when adding a worker, or refer to the Installation documentation for more details.
2. Ensure the NVIDIA driver, Docker and NVIDIA Container Toolkit are installed on the worker node.
3. (Optional) A CPU node for hosting the GPUStack server. The GPUStack server does not require a GPU and can run on a CPU-only machine. Docker must be installed. Docker Desktop (for Windows and macOS) is also supported. If no dedicated CPU node is available, the GPUStack server can be installed on the same machine as a GPU worker node.
4. Only Linux is supported for GPUStack worker nodes. If you use Windows, consider using WSL2 and avoid using Docker Desktop. macOS is not supported for GPUStack worker nodes.

Install GPUStack

Run the following command to install and start the GPUStack server using Docker:

sudo docker run -d --name gpustack \
    --restart unless-stopped \
    -p 80:80 \
    --volume gpustack-data:/var/lib/gpustack \
    gpustack/gpustack

<details> <summary>Alternative: Use Quay Container Registry Mirror</summary>

If you cannot pull images from Docker Hub or the download is very slow, you can use our Quay.io mirror by pointing your registry to quay.io:

sudo docker run -d --name gpustack \
    --restart unless-stopped \
    -p 80:80 \
    --volume gpustack-data:/var/lib/gpustack \
    quay.io/gpustack/gpustack \
    --system-default-container-registry quay.io

Check the GPUStack startup logs:

sudo docker logs -f gpustack

After GPUStack starts, run the following command to get the default admin password:

sudo docker exec gpustack cat /var/lib/gpustack/initial_admin_password

Open your browser and navigate to http://your_host_ip to access the GPUStack UI. Use the default username admin and the password you retrieved above to log in.

Deploy a Model

1. Navigate to the Catalog page in the GPUStack UI.

1. Select the Qwen3.5-0.8B model from the list of available models.

1. After the deployment compatibility checks pass, click the Save button to deploy the model.

1. GPUStack will start downloading the model files and deploying the model. When the deployment status shows Running, the model has been deployed successfully.

1. Click Playground - Chat in the navigation menu, check that the model qwen3.5-0.8b is selected from the top-right Model dropdown. Now you can chat with the model in the UI playground.

Build

1. Install Docker.

1. Run make package.

Quick Start

Use the model via API

1. Navigate to the Access Control > API Keys page, then click the New API Key button.

1. Fill in the Name and click the Save button.

1. Copy the generated API key and save it somewhere safe. Please note that you can only see it once on creation.

1. You can now use the API key to access the OpenAI-compatible API endpoints provided by GPUStack. For example, use curl as the following:

```bash

Replace `your_api_key` and `your_gpustack_server_url`

with your actual API key and GPUStack server URL.

export GPUSTACK_API_KEY=your_api_key curl http://your_gpustack_server_url/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $GPUSTACK_API_KEY" \ -d '{ "model": "qwen3.5-0.8b", "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": "Tell me a joke." } ], "stream": true }' ```