Intermediate · Generative AI
Auto-regressive model
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A model that predicts future outputs based on its own past predictions, commonly used in language models to generate sequences.
Technical Definition
A model that predicts future outputs based on its own past predictions, commonly used in language models to generate sequences.
How it works
An auto-regressive model generates a sequence of outputs where each prediction is dependent on the previously generated predictions. Language models like Transformers are typically auto-regressive, predicting the next word based on the preceding words. This sequential dependency allows them to generate coherent and contextually relevant text.
Related Concepts
- Generative AI — An AI field focused on creating models that can generate novel and complex content, such as text, images, audio, and video, that is coherent and original.