Home › Glossary › Generative AI › QLoRA

Advanced · Generative AI

QLoRA

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Fine-tuning method that adds LoRA adapters on top of a 4-bit quantized base model, letting 65B LLMs be trained on a single 48 GB GPU.

Technical Definition

Fine-tuning method that adds LoRA adapters on top of a 4-bit quantized base model, letting 65B LLMs be trained on a single 48 GB GPU.

How it works

QLoRA (Dettmers et al., 2023) combines three innovations: 4-bit NormalFloat quantization of the frozen base weights, double quantization of the quantization constants, and paged optimizers to handle gradient spikes. Only the LoRA adapters are trained in BF16. Quality matches 16-bit LoRA fine-tuning, making large-model fine-tuning accessible on consumer hardware.

Related Concepts

  • Fine-Tuning — Adapting a pre-trained model to a specific task by continuing training on a smaller, task-specific dataset.
  • LoRA (Low-Rank Adaptation) — A parameter-efficient fine-tuning method injecting small trainable matrices into frozen pre-trained layers.
  • Quantization — Reducing numerical precision of model weights (e.g., 32-bit to 4-bit) to shrink size and speed up inference.
  • PEFT (Parameter-Efficient Fine-Tuning) — A family of methods that adapt large pretrained models by training only a small fraction of parameters, drastically cutting compute and storage.