Advanced · Data
Gradient boosting
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A machine learning method that builds models sequentially, with each new model correcting the errors of the previous ones.
Technical Definition
A machine learning method that builds models sequentially, with each new model correcting the errors of the previous ones.
How it works
Gradient boosting is a powerful ensemble technique where models are added one after another. Each new model focuses on the 'pseudo-residuals' (errors) of the combined ensemble, effectively boosting the performance of the overall model.
Related Concepts
- Loss Function — A mathematical function that measures how far the model's predictions are from the actual values, guiding the learning process.
- Machine Learning — A field of AI where systems learn patterns from data instead of following hard-coded rules.
- Boosting — A machine learning technique that trains models sequentially, with each new model correcting the errors of the previous ones, primarily to reduce bias.
- Ensemble learning — Combining multiple machine learning models to improve overall prediction accuracy and robustness.