Intermediate · Evaluation

Bias-Variance Trade-off

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. The conflict in simultaneously minimizing two sources of error that prevent models from generalizing beyond their training data.

Technical Definition

The conflict in simultaneously minimizing two sources of error that prevent models from generalizing beyond their training data.

How it works

Bias is the error due to overly simplistic assumptions in the learning algorithm, leading to underfitting. Variance is the error due to excessive complexity in the learning algorithm, making it too sensitive to the training data and causing overfitting. Reducing one typically increases the other; thus, finding an optimal balance is crucial for building robust and generalizable models. This trade-off is fundamental to understanding model performance.

Related Concepts

• Overfitting — When a model learns noise and specific patterns in training data too well, causing it to perform poorly on new, unseen data.
• Generalization — A model's ability to perform well on new, unseen data — not just its training set.
• Underfitting — A phenomenon where a model is too simple to capture the underlying patterns in the training data, resulting in poor performance on both training and new data.