Intermediate · Fundamentals
Boosting
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A machine learning technique that trains models sequentially, with each new model correcting the errors of the previous ones, primarily to reduce bias.
Technical Definition
A machine learning technique that trains models sequentially, with each new model correcting the errors of the previous ones, primarily to reduce bias.
How it works
Boosting is an ensemble method where models are trained one after another. Each subsequent model focuses on correcting the mistakes made by the earlier models. This sequential correction process is particularly effective at reducing bias and improving the overall accuracy of the model.
Related Concepts
- Ensemble learning — Combining multiple machine learning models to improve overall prediction accuracy and robustness.
- Gradient boosting — A machine learning method that builds models sequentially, with each new model correcting the errors of the previous ones.
- Bagging — An ensemble technique where models are trained on random subsets of data sampled with replacement to reduce variance.