ChatTTS Agent工作流

Install requirements

#### 1. Install Directly

pip install --upgrade -r requirements.txt

#### 2. Install from conda

conda create -n chattts python=3.11
conda activate chattts
pip install -r requirements.txt

#### Optional: Install vLLM (Linux only)

pip install safetensors vllm==0.2.7 torchaudio

#### Unrecommended Optional: Install TransformerEngine if using NVIDIA GPU (Linux only) > [!Warning] > DO NOT INSTALL! > The adaptation of TransformerEngine is currently under development and CANNOT run properly now. > Only install it on developing purpose. See more details on at #672 #676

[!Note] The installation process is very slow.

pip install git+https://github.com/NVIDIA/TransformerEngine.git@stable

#### Unrecommended Optional: Install FlashAttention-2 (mainly NVIDIA GPU) > [!Warning] > DO NOT INSTALL! > Currently the FlashAttention-2 will slow down the generating speed according to this issue. > Only install it on developing purpose.

[!Note] See supported devices at the Hugging Face Doc.

pip install flash-attn --no-build-isolation

Installation

1. Install the stable version from PyPI

pip install ChatTTS

2. Install the latest version from GitHub

pip install git+https://github.com/2noise/ChatTTS

3. Install from local directory in dev mode

pip install -e .

Quick Start

Make sure you are under the project root directory when you execute these commands below.

#### 1. Launch WebUI

python examples/web/webui.py

#### 2. Infer by Command Line > It will save audio to ./output_audio_n.mp3

python examples/cmd/run.py "Your text 1." "Your text 2."

Basic Usage

import ChatTTS
import torch
import torchaudio

chat = ChatTTS.Chat()
chat.load(compile=False) # Set to True for better performance

texts = ["PUT YOUR 1st TEXT HERE", "PUT YOUR 2nd TEXT HERE"]

wavs = chat.infer(texts)

for i in range(len(wavs)):
    """
    In some versions of torchaudio, the first line works but in other versions, so does the second line.
    """
    try:
        torchaudio.save(f"basic_output{i}.wav", torch.from_numpy(wavs[i]).unsqueeze(0), 24000)
    except:
        torchaudio.save(f"basic_output{i}.wav", torch.from_numpy(wavs[i]), 24000)

Advanced Usage

```python ###################################

Sample a speaker from Gaussian.

rand_spk = chat.sample_random_speaker() print(rand_spk) # save it for later timbre recovery

params_infer_code = ChatTTS.Chat.InferCodeParams( spk_emb = rand_spk, # add sampled speaker temperature = .3, # using custom temperature top_P = 0.7, # top P decode top_K = 20, # top K decode )

###################################

FAQ

#### 1. How much VRAM do I need? How about infer speed? For a 30-second audio clip, at least 4GB of GPU memory is required. For the 4090 GPU, it can generate audio corresponding to approximately 7 semantic tokens per second. The Real-Time Factor (RTF) is around 0.3.

2. Model stability is not good enough, with issues such as multi speakers or poor audio quality.

This is a problem that typically occurs with autoregressive models (for bark and valle). It's generally difficult to avoid. One can try multiple samples to find a suitable result.

3. Besides laughter, can we control anything else? Can we control other emotions?

In the current released model, the only token-level control units are [laugh], [uv_break], and [lbreak]. In future versions, we may open-source models with additional emotional control capabilities.