Advanced · Research
Candidate sampling
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A training optimization technique that reduces computational cost by only calculating probabilities for positive labels and a random sample of negative labels.
Technical Definition
A training optimization technique that reduces computational cost by only calculating probabilities for positive labels and a random sample of negative labels.
How it works
Candidate sampling is an efficiency technique used during model training, particularly when dealing with a large number of possible output classes. Instead of computing the loss for all possible negative classes, it computes the loss for the true positive class and a randomly selected subset of negative classes. This significantly speeds up training without drastically compromising model performance.
Related Concepts
- Loss Function — A mathematical function that measures how far the model's predictions are from the actual values, guiding the learning process.
- Softmax Function — A function that converts a vector of raw scores into a probability distribution where all values sum to one.