Intermediate · Data
Gradient boosted (decision) trees (GBT)
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. An ensemble machine learning technique that combines multiple decision trees sequentially, where each new tree corrects the errors of the previous ones.
Technical Definition
An ensemble machine learning technique that combines multiple decision trees sequentially, where each new tree corrects the errors of the previous ones.
How it works
Gradient Boosted Trees (GBT), often referred to as Gradient Boosted Decision Trees (GBDT), is a powerful machine learning algorithm used for both regression and classification tasks. It builds an additive model where each new tree is trained to predict the residual errors of the combined ensemble of previous trees, using gradient descent optimization. This sequential process allows GBT models to achieve high accuracy by iteratively improving predictions.
Related Concepts
- Gradient Descent — An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease of the loss function.
- Machine Learning — A field of AI where systems learn patterns from data instead of following hard-coded rules.
- Ensemble learning — Combining multiple machine learning models to improve overall prediction accuracy and robustness.
- Decision tree — A decision tree is a supervised learning model that uses a tree-like structure of decisions and their possible consequences.