Beginner · Fundamentals

Parameters

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. The internal variables — weights and biases — that a model learns from data.

Technical Definition

The internal variables — weights and biases — that a model learns from data.

How it works

Parameter count is a rough proxy for model capacity. GPT-2 had 1.5B parameters; modern frontier models reach trillions. Parameters are distinct from hyperparameters (learning rate, batch size), which are set by the engineer rather than learned.

Related Concepts

• Hyperparameter Tuning — The process of finding optimal configuration values that control model training, such as learning rate, batch size, and architecture choices.
• Weights — The learnable numerical parameters that determine how a neural network transforms its inputs.
• Bias (model parameter) — A learnable constant added to a neuron's weighted input that shifts the activation function.