Intermediate · Data
Entropy
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A measure of the unpredictability or randomness in a probability distribution.
Technical Definition
A measure of the unpredictability or randomness in a probability distribution.
How it works
In information theory, entropy quantifies the amount of uncertainty or randomness present in a set of data or a probability distribution. A system with high entropy is unpredictable, meaning each outcome is roughly equally likely. Low entropy indicates predictability, where some outcomes are much more probable than others.