Intermediate · Neural Networks

Parameter-Efficient Fine-Tuning (PEFT)

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. A family of techniques that adapt large pre-trained models to new tasks by updating only a small subset of their parameters, saving computation.

Technical Definition

A family of techniques that adapt large pre-trained models to new tasks by updating only a small subset of their parameters, saving computation.

How it works

PEFT methods, including techniques like LoRA and prompt tuning, address the challenge of costly full model fine-tuning for large models. By judiciously selecting and updating only a small fraction of the model's parameters, PEFT maintains high performance while significantly reducing computational resources, storage, and training time. This makes customization of LLMs more practical.

Related Concepts

• Fine-Tuning — Adapting a pre-trained model to a specific task by continuing training on a smaller, task-specific dataset.
• Transfer Learning — Leveraging knowledge from a model trained on one task to improve performance on a different but related task.
• LoRA (Low-Rank Adaptation) — A parameter-efficient fine-tuning method injecting small trainable matrices into frozen pre-trained layers.
• Prompt Tuning — A parameter-efficient fine-tuning technique that learns a small, task-specific soft prompt (a sequence of learnable tokens) to steer a frozen LLM.