Beginner · Research
Early stopping
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A regularization technique used in iterative training to stop model learning when performance on a validation set starts to degrade.
Technical Definition
A regularization technique used in iterative training to stop model learning when performance on a validation set starts to degrade.
How it works
Early stopping is a method to prevent overfitting during the training of iterative machine learning models, such as neural networks trained with gradient descent. It involves monitoring the model's performance on a separate validation dataset and halting training when this performance begins to worsen, indicating that the model is starting to memorize the training data rather than generalize.
Related Concepts
- Gradient Descent — An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease of the loss function.
- Overfitting — When a model learns noise and specific patterns in training data too well, causing it to perform poorly on new, unseen data.
- Regularization — Techniques that constrain a model's complexity to prevent overfitting and improve generalization to unseen data.