CrisperWhisper — AI 语音识别工具中文文档

2.1 Prerequisites

• Python: 3.10
• PyTorch: 2.0
• NVIDIA Libraries: cuBLAS 11.x and cuDNN 8.x (for GPU execution)

4.1 Prerequisites

Make sure you have followed the Setup ⚙️ instructions above and have the crisperWhisper environment activated.

2. Setup ⚙️

2.2 Environment Setup

1. Clone the Repository:

    git clone https://github.com/nyrahealth/CrisperWhisper.git
    cd CrisperWhisper
    

2. Create Python Environment:

    conda create --name crisperWhisper python=3.10
    conda activate crisperWhisper
    

3. Install Dependencies:

    pip install -r requirements.txt
    

4. Additional Installations: Follow OpenAI's instructions to install additional dependencies like ffmpeg and rust: Whisper Setup.

3. Usage

Here's how to use CrisperWhisper in your Python scripts: First install our custom transformers fork for the most accurate timestamps:

pip install git+https://github.com/nyrahealth/transformers.git@crisper_whisper

3.1 Usage with 🤗 transformers

First make sure that you have a huggingface account and accept the licensing of the model. Grab your huggingface access token and login so you are certainly able to download the model.

huggingface-cli login

import os
import sys
import torch

from datasets import load_dataset
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
from utils import adjust_pauses_for_hf_pipeline_output



device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32

model_id = "nyrahealth/CrisperWhisper"

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True
)
model.to(device)

processor = AutoProcessor.from_pretrained(model_id)

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    chunk_length_s=30,
    batch_size=16,
    return_timestamps='word',
    torch_dtype=torch_dtype,
    device=device,
)

dataset = load_dataset("distil-whisper/librispeech_long", "clean", split="validation")
sample = dataset[0]["audio"]
hf_pipeline_output = pipe(sample)
crisper_whisper_result = adjust_pauses_for_hf_pipeline_output(hf_pipeline_output)
print(crisper_whisper_result)

3.2 Usage with faster whisper

We also provide a converted model to be compatible with faster whisper. However, due to the different implementation of the timestamp calculation in faster whisper or more precisely CTranslate2 the timestamp accuracy can not be guaranteed.

First make sure that you have a huggingface account and accept the licensing of the model. Grab your huggingface access token and login so you are certainly able to download the model.

huggingface-cli login

```python from faster_whisper import WhisperModel from datasets import load_dataset faster_whisper_model = 'nyrahealth/faster_CrisperWhisper'

3.3 Commandline usage

First make sure that you have a huggingface account and accept the licensing of the model. Grab your huggingface access token and login so you are certainly able to download the model.

    huggingface-cli login
 

To transcribe an audio file, use the following command:

python transcribe.py --f <path_to_audio_file>