Intermediate · Generative AI

Instruction Tuning

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Fine-tuning a pretrained language model on examples of instructions paired with desired responses.

Technical Definition

Fine-tuning a pretrained language model on examples of instructions paired with desired responses.

How it works

Raw pretrained LLMs predict the next token but don't naturally follow commands. Instruction tuning on datasets like FLAN or Alpaca teaches them to interpret 'Summarize this in three bullets' as a task to perform. It is usually followed by RLHF or DPO to align outputs with human preferences.

Related Concepts

• Large Language Model (LLM) — A massive neural network trained on vast text corpora to understand and generate human language with remarkable fluency.
• Fine-Tuning — Adapting a pre-trained model to a specific task by continuing training on a smaller, task-specific dataset.
• RLHF (Reinforcement Learning from Human Feedback) — A technique that aligns LLM outputs with human preferences by training a reward model from human comparisons.