Beginner · Research

Random Search

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Sampling hyperparameter configurations at random instead of testing every combination.

Technical Definition

Sampling hyperparameter configurations at random instead of testing every combination.

How it works

Surprisingly effective: in high-dimensional spaces, only a few hyperparameters typically matter, and random sampling explores those dimensions more efficiently than grid search. A strong default baseline before reaching for fancier methods.

Related Concepts

• Hyperparameter Tuning — The process of finding optimal configuration values that control model training, such as learning rate, batch size, and architecture choices.
• Grid Search — Exhaustively trying every combination of a predefined set of hyperparameter values.
• Bayesian Optimization — An adaptive hyperparameter search that uses a probabilistic model to choose the next configuration to try.