Intermediate · Generative AI
Emergent behavior
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. Capabilities or behaviors in large language models that were not explicitly programmed or trained for.
Technical Definition
Capabilities or behaviors in large language models that were not explicitly programmed or trained for.
How it works
Emergent behavior refers to abilities that appear in large language models (LLMs) as they scale up, which are not present in smaller models. These behaviors arise spontaneously and are not directly taught through the training data or objectives. Examples include performing arithmetic or answering questions on topics outside the direct training set.
Related Concepts
- Large Language Model (LLM) — A massive neural network trained on vast text corpora to understand and generate human language with remarkable fluency.
- Few-Shot Learning — Training a model to recognize new patterns from just a handful of labeled examples.
- Scaling Laws — Empirical power-law relationships predicting how LLM loss decreases as model size, dataset size, and compute increase together.