Intermediate · Safety

Data Leakage

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. When information that wouldn't be available at inference time accidentally influences training.

Technical Definition

When information that wouldn't be available at inference time accidentally influences training.

How it works

Examples: future data leaking into training features, target variables included as inputs, test data overlapping with training. Leakage produces models that look great offline and fail in production. Detecting it requires careful pipeline design and rigorous evaluation.

Related Concepts

• Overfitting — When a model learns noise and specific patterns in training data too well, causing it to perform poorly on new, unseen data.
• Training Data — The portion of a dataset used to fit a model's parameters.
• Reproducibility — The ability for others to re-run an experiment and obtain the same results.