ML//training//fine-tuning//QLoRA

QLoRA is quantized LoRA: the base model is loaded in 4-bit precision and the LoRA adapters are trained in 16-bit.

QLoRA is quantized LoRA: the base model is loaded in 4-bit precision and the LoRA adapters are trained in 16-bit.

It fine-tunes a 65B model on a single 48GB GPU, which brought model customization to consumer hardware: anyone with a decent GPU can customize frontier-class models. Combined with FSDP, QLoRA scales across several GPUs for even larger models.