Llama3情绪分类工具

Setup

This project uses LLaMA-Factory as a pip dependency — no framework code is vendored here:

git clone https://github.com/DaoyuanLi2816/llama3-emotion-lora.git
cd llama3-emotion-lora
pip install -r requirements.txt   # llamafactory[metrics]; bitsandbytes for 16 GB GPUs
huggingface-cli login             # the Llama3 weights are gated

Usage

Fine-tune Llama3-8b with LoRA on the emotion dataset (hyperparameters exactly as reported in Table 2 — Adam lr 5e-5, cosine schedule, batch 5 × grad-accum 4, 3 epochs, LoRA rank 8, max length 512, fp16, FlashAttention-2):

llamafactory-cli train config/emotion_llama3_lora.yaml

Generate predictions for the 2,000-sample test split with the trained adapter, then score them:

llamafactory-cli train config/emotion_llama3_predict.yaml
python scripts/evaluate.py saves/llama3-8b-emotion-lora/predict/generated_predictions.jsonl

The fp16 run fits a 24 GB GPU; on 16 GB cards add quantization_bit: 4 (QLoRA) to the training config.

Experiment Settings

The Llama3-8b model's hyperparameters are set as follows:

The model is trained using the Adam optimizer, known for its adaptive learning rate capabilities. A cosine learning rate schedule is employed to adjust the learning rate during training. The batch size is set to 5, with gradient accumulation over 4 steps to optimize memory usage. The model is trained for 3 epochs, with the FP16 precision format used to save GPU memory while maintaining performance. The LoRA rank of 8 indicates the order of the low-rank matrix used in the adaptation process.