Beginner · Fundamentals
Batch size
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. The number of training examples processed in one iteration of a machine learning algorithm's update cycle.
Technical Definition
The number of training examples processed in one iteration of a machine learning algorithm's update cycle.
How it works
Batch size refers to the number of training examples used in one forward and backward pass of a neural network. A larger batch size can lead to more stable gradients but requires more memory, while a smaller batch size introduces more noise but can sometimes lead to faster convergence or escape local minima. Common strategies include stochastic gradient descent (batch size 1) and mini-batch (a small to moderate number of examples).
Related Concepts
- Gradient Descent — An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease of the loss function.