Intermediate · Generative AI
In-context learning
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A technique where a language model learns to perform a task by conditioning its output on examples provided within the input prompt.
Technical Definition
A technique where a language model learns to perform a task by conditioning its output on examples provided within the input prompt.
How it works
In-context learning, often referred to as few-shot prompting, is a capability of large language models. Instead of explicit fine-tuning, the model is presented with a few examples of the desired task within the prompt itself. It then uses these examples to understand the pattern and generate appropriate outputs for new, similar inputs.
Related Concepts
- Large Language Model (LLM) — A massive neural network trained on vast text corpora to understand and generate human language with remarkable fluency.
- Prompt Engineering — The art of crafting effective input instructions to guide LLM behavior without changing model weights.
- Few-Shot Learning — Training a model to recognize new patterns from just a handful of labeled examples.
- Generative AI — An AI field focused on creating models that can generate novel and complex content, such as text, images, audio, and video, that is coherent and original.