Intermediate · Fundamentals
Bootstrap aggregating
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. An ensemble technique that trains multiple models independently and averages their predictions to reduce variance, commonly known as bagging.
Technical Definition
An ensemble technique that trains multiple models independently and averages their predictions to reduce variance, commonly known as bagging.
How it works
Bootstrap aggregating, or bagging, is an ensemble meta-heuristic that improves model stability and accuracy. It works by training several models independently on different bootstrap samples of the data and then aggregating their predictions (e.g., by averaging). This process is effective at reducing variance.
Related Concepts
- Ensemble learning — Combining multiple machine learning models to improve overall prediction accuracy and robustness.
- Random forest — An ensemble machine learning method that builds multiple decision trees to improve prediction accuracy and control overfitting.
- Bagging — An ensemble technique where models are trained on random subsets of data sampled with replacement to reduce variance.