Intermediate · Research
Decision forest
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A decision forest is an ensemble of multiple decision trees, combining their predictions for improved accuracy and robustness.
Technical Definition
A decision forest is an ensemble of multiple decision trees, combining their predictions for improved accuracy and robustness.
How it works
A decision forest, also known as a random forest or gradient boosted trees, is an ensemble learning method that constructs multiple decision trees during training. The final prediction is determined by averaging (for regression) or voting (for classification) the predictions of individual trees. This ensemble approach typically leads to better generalization and reduces overfitting compared to a single decision tree.
Related Concepts
- Ensemble learning — Combining multiple machine learning models to improve overall prediction accuracy and robustness.
- Random forest — An ensemble machine learning method that builds multiple decision trees to improve prediction accuracy and control overfitting.
- Decision tree — A decision tree is a supervised learning model that uses a tree-like structure of decisions and their possible consequences.
- Gradient boosted (decision) trees (GBT) — An ensemble machine learning technique that combines multiple decision trees sequentially, where each new tree corrects the errors of the previous ones.