MockingBird — AI 语音合成工具中文文档

1. Install Requirements

#### 1.1 General Setup > Follow the original repo to test if you got all environment ready. Python 3.7 or higher is needed to run the toolbox.

Install PyTorch. > If you get an ERROR: Could not find a version that satisfies the requirement torch==1.9.0+cu102 (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2 ) This error is probably due to a low version of python, try using 3.9 and it will install successfully Install ffmpeg. Run pip install -r requirements.txt to install the remaining necessary packages. > The recommended environment here is Repo Tag 0.0.1 Pytorch1.9.0 with Torchvision0.10.0 and cudatoolkit10.2 requirements.txt webrtcvad-wheels because requirements. txt was exported a few months ago, so it doesn't work with newer versions Install webrtcvad pip install webrtcvad-wheels(If you need)

or - install dependencies with conda or mamba

 env create -n env_name -f env.yml

 env create -n env_name -f env.yml

will create a virtual environment where necessary dependencies are installed. Switch to the new environment by conda activate env_name and enjoy it. > env.yml only includes the necessary dependencies to run the project，temporarily without monotonic-align. You can check the official website to install the GPU version of pytorch.

#### 1.2 Setup with a M1 Mac > The following steps are a workaround to directly use the original demo_toolbox.pywithout the changing of codes. > > Since the major issue comes with the PyQt5 packages used in demo_toolbox.py not compatible with M1 chips, were one to attempt on training models with the M1 chip, either that person can forgo demo_toolbox.py, or one can try the web.py in the project.

##### 1.2.1 Install PyQt5, with ref here. Create and open a Rosetta Terminal, with ref here. Use system Python to create a virtual environment for the project

    /usr/bin/python3 -m venv /PathToMockingBird/venv
    source /PathToMockingBird/venv/bin/activate
    

    pip install --upgrade pip
    pip install pyqt5
    

Both packages seem to be unique to this project and are not seen in the original Real-Time Voice Cloning project. When installing with pip install, both packages lack wheels so the program tries to directly compile from c code and could not find Python.h.

• Install pyworld
• brew install python Python.h can come with Python installed by brew
• export CPLUS_INCLUDE_PATH=/opt/homebrew/Frameworks/Python.framework/Headers The filepath of brew-installed Python.h is unique to M1 MacOS and listed above. One needs to manually add the path to the environment variables.
• pip install pyworld that should do.

Installctc-segmentation > Same method does not apply to ctc-segmentation, and one needs to compile it from the source code on github. git clone https://github.com/lumaku/ctc-segmentation.git cd ctc-segmentation source /PathToMockingBird/venv/bin/activate If the virtual environment hasn't been deployed, activate it. cythonize -3 ctc_segmentation/ctc_segmentation_dyn.pyx /usr/bin/arch -x86_64 python setup.py build Build with x86 architecture. * /usr/bin/arch -x86_64 python setup.py install --optimize=1 --skip-buildInstall with x86 architecture.

##### 1.2.3 Other dependencies /usr/bin/arch -x86_64 pip install torch torchvision torchaudio Pip installing PyTorch as an example, articulate that it's installed with x86 architecture pip install ffmpeg Install ffmpeg * pip install -r requirements.txt Install other requirements.

##### 1.2.4 Run the Inference Time (with Toolbox) > To run the project on x86 architecture. ref. vim /PathToMockingBird/venv/bin/pythonM1 Create an executable file pythonM1 to condition python interpreter at /PathToMockingBird/venv/bin. Write in the following content:

    #!/usr/bin/env zsh
    mydir=${0:a:h}
    /usr/bin/arch -x86_64 $mydir/python "$@"
    

Quick Start

[DEMO VIDEO](https://www.bilibili.com/video/BV17Q4y1B7mY/)

Reference

This repository is forked from Real-Time-Voice-Cloning which only support English.

F Q&A

#### 1.Where can I download the dataset? <div class="rdm-tbl-wrap"><table class="rdm-tbl"><thead><tr><th>Dataset</th><th>Original Source</th><th>Alternative Sources</th></tr></thead><tbody><tr><td>aidatatang_200zh</td><td>OpenSLR</td><td>Google Drive</td></tr><tr><td>magicdata</td><td>OpenSLR</td><td>Google Drive (Dev set)</td></tr><tr><td>aishell3</td><td>OpenSLR</td><td>Google Drive</td></tr><tr><td>data_aishell</td><td>OpenSLR</td><td></td></tr></tbody></table></div> > After unzip aidatatang_200zh, you need to unzip all the files under aidatatang_200zh\corpus\train

#### 2.What is<datasets_root>? If the dataset path is D:\data\aidatatang_200zh,then <datasets_root> isD:\data

#### 3.Not enough VRAM Train the synthesizer：adjust the batch_size in synthesizer/hparams.py

//Before
tts_schedule = [(2,  1e-3,  20_000,  12),   # Progressive training schedule
                (2,  5e-4,  40_000,  12),   # (r, lr, step, batch_size)
                (2,  2e-4,  80_000,  12),   #
                (2,  1e-4, 160_000,  12),   # r = reduction factor (# of mel frames
                (2,  3e-5, 320_000,  12),   #     synthesized for each decoder iteration)
                (2,  1e-5, 640_000,  12)],  # lr = learning rate
//After
tts_schedule = [(2,  1e-3,  20_000,  8),   # Progressive training schedule
                (2,  5e-4,  40_000,  8),   # (r, lr, step, batch_size)
                (2,  2e-4,  80_000,  8),   #
                (2,  1e-4, 160_000,  8),   # r = reduction factor (# of mel frames
                (2,  3e-5, 320_000,  8),   #     synthesized for each decoder iteration)
                (2,  1e-5, 640_000,  8)],  # lr = learning rate

Train Vocoder-Preprocess the data：adjust the batch_size in synthesizer/hparams.py ``` //Before