Advanced · Research
Double descent
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A phenomenon where model test error is low for both very simple and very complex models, but high for models with intermediate complexity.
Technical Definition
A phenomenon where model test error is low for both very simple and very complex models, but high for models with intermediate complexity.
How it works
Double descent describes an intriguing pattern in machine learning where model performance, as measured by test error, improves, then worsens, and then improves again as model complexity increases. Typically, performance degrades around the point where model capacity equals the amount of training data (the interpolation threshold), contradicting classical overfitting theories.
Related Concepts
- Overfitting — When a model learns noise and specific patterns in training data too well, causing it to perform poorly on new, unseen data.
- Generalization — A model's ability to perform well on new, unseen data — not just its training set.