Beginner · Fundamentals

Cost Function

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Another name for the loss function, typically averaged over the entire training set.

Technical Definition

Another name for the loss function, typically averaged over the entire training set.

How it works

Loss is often used per-example, cost per-dataset, but the terms are used interchangeably. The cost function defines what the model is optimizing — choose it carefully: cross-entropy for classification, MSE for regression, custom losses for ranking or generation.

Related Concepts

• Loss Function — A mathematical function that measures how far the model's predictions are from the actual values, guiding the learning process.
• Training — The process of adjusting a model's parameters so it learns patterns from labeled or unlabeled data.
• Optimization — The mathematical process of finding parameter values that minimize a loss function.