Beginner · Data

Test Data

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. A separate, untouched dataset used only at the end to estimate real-world performance.

Technical Definition

A separate, untouched dataset used only at the end to estimate real-world performance.

How it works

The test set is the final exam — touched once, after all training and tuning are complete. Repeatedly evaluating on the test set leaks information and inflates reported metrics. Production teams often hold out additional 'lockbox' or temporal test sets.

Related Concepts

• Training Data — The portion of a dataset used to fit a model's parameters.
• Validation Data — A held-out subset used to tune hyperparameters and detect overfitting during training.
• Generalization — A model's ability to perform well on new, unseen data — not just its training set.