Advanced · Neural Networks

Prompt Tuning

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. A parameter-efficient fine-tuning technique that learns a small, task-specific soft prompt (a sequence of learnable tokens) to steer a frozen LLM.

Technical Definition

A parameter-efficient fine-tuning technique that learns a small, task-specific soft prompt (a sequence of learnable tokens) to steer a frozen LLM.

How it works

Prompt tuning is a method within PEFT that optimizes continuous vectors acting as a 'soft prompt' to condition a pre-trained LLM for a specific task. Unlike traditional fine-tuning that updates model weights, the core LLM remains unaltered, saving significant computational resources. The learned soft prompt effectively guides the large model to produce the desired output for new tasks.

Related Concepts

• Fine-Tuning — Adapting a pre-trained model to a specific task by continuing training on a smaller, task-specific dataset.
• LoRA (Low-Rank Adaptation) — A parameter-efficient fine-tuning method injecting small trainable matrices into frozen pre-trained layers.
• Parameter-Efficient Fine-Tuning (PEFT) — A family of techniques that adapt large pre-trained models to new tasks by updating only a small subset of their parameters, saving computation.