Intermediate · Evaluation
Cross-entropy
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. Cross-entropy measures the difference between two probability distributions, commonly used for multi-class classification loss.
Technical Definition
Cross-entropy measures the difference between two probability distributions, commonly used for multi-class classification loss.
How it works
Cross-entropy is a metric used to quantify the difference between two probability distributions, often comparing the predicted probabilities of a model against the true probabilities. It is widely used as a loss function in multi-class classification problems. Lower cross-entropy values indicate a better fit between the predicted and true distributions.
Related Concepts
- Classification — A supervised learning task where the model assigns inputs to discrete categories.