Intermediate · Data
Gini impurity
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A measure of how well a function separates data into distinct classes, often used in decision trees to evaluate the quality of splits.
Technical Definition
A measure of how well a function separates data into distinct classes, often used in decision trees to evaluate the quality of splits.
How it works
Gini impurity is a metric used in constructing decision trees to quantify the level of misclassification that would occur if a randomly selected element were labeled according to the distribution of elements in the subset. A lower Gini impurity indicates a more homogeneous subset, meaning it is purer with respect to the class labels. It is calculated as 1 minus the sum of the squares of the probabilities of each class.
Related Concepts
- Classification — A supervised learning task where the model assigns inputs to discrete categories.
- Decision tree — A decision tree is a supervised learning model that uses a tree-like structure of decisions and their possible consequences.
- Entropy — A measure of the unpredictability or randomness in a probability distribution.
- Information gain — A metric used in decision trees to measure the expected reduction in entropy achieved by splitting data on a particular feature.