onediff Agent工作流

Installation

#### 0. OS and GPU Compatibility - Linux - If you want to use onediff on Windows, please use it under WSL. - The guide to install onediff in WSL2. - NVIDIA GPUs - Compatibility with Nvidia GPUs.

#### 1. Install torch and diffusers Note: You can choose the latest versions you want for diffusers or transformers.

python3 -m pip install "torch" "transformers==4.27.1" "diffusers[torch]==0.19.3"

#### 2. Install a compiler backend When considering the choice between OneFlow and Nexfort, either one is optional, and only one is needed.

• For DiT structural models or H100 devices, it is recommended to use Nexfort.

• For all other cases, it is recommended to use OneFlow. Note that optimizations within OneFlow will gradually transition to Nexfort in the future.

##### Nexfort Install Nexfort is Optional. The detailed introduction of Nexfort is here.

python3 -m  pip install -U torch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 torchao==0.1
python3 -m  pip install -U nexfort

##### OneFlow Install OneFlow is Optional. > NOTE: We have updated OneFlow frequently for onediff, so please install OneFlow by the links below.

• CUDA 11.8

For NA/EU users

  python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu118
  

For CN users

  python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu118
  

<details> <summary> Click to get OneFlow packages for other CUDA versions. </summary>

• CUDA 12.1

For NA/EU users

  python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu122
  

For CN users

  python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu122
  

• CUDA 12.2

For NA/EU users

  python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu122
  

  python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu122
  

</details>

3. Install onediff

- From PyPI

python3 -m pip install --pre onediff

git clone https://github.com/siliconflow/onediff.git

cd onediff && python3 -m pip install -e .

install for dev

cd onediff && python3 -m pip install -e '.[dev]'

Acceleration for production environment

#### PyTorch Module compilation - compilation with oneflow_compile #### Avoid compilation time for new input shape - Support Multi-resolution input #### Avoid compilation time for online serving Compile and save the compiled result offline, then load it online for serving - Save and Load the compiled graph - Compile at one device(such as device 0), then use the compiled result to other device(such as device 1~7). Change device of the compiled graph to do multi-process serving #### Distributed Run If you want to do distributed inference, you can use onediff's compiler to do single-device acceleration in a distributed inference engine such as xDiT

Performance comparison

<img src="imgs/replace_a100.png" height="400">

#### SDXL E2E time - Model stabilityai/stable-diffusion-xl-base-1.0; - Image size 1024*1024, batch size 1, steps 30; - NVIDIA A100 80G SXM4;

<img src="imgs/0_12_sdxl.png" height="400">

#### SVD E2E time - Model stabilityai/stable-video-diffusion-img2vid-xt; - Image size 576*1024, batch size 1, steps 25, decoder chunk size 5; - NVIDIA A100 80G SXM4;

<img src="imgs/0_12_svd.png" height="400">

Note that we haven't got a way to run SVD with TensorRT on Feb 29 2024.