Beginner · Neural Networks

Weights

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. The learnable numerical parameters that determine how a neural network transforms its inputs.

Technical Definition

The learnable numerical parameters that determine how a neural network transforms its inputs.

How it works

Each connection between neurons stores a weight; together they encode everything the model has learned. During training, weights are updated via gradient descent to minimize loss. A modern LLM may contain hundreds of billions of weights, while a small image classifier may have only a few million.

Related Concepts

• Neural Network — A computing system inspired by biological neural networks that learns patterns from data through interconnected layers of nodes.
• Gradient Descent — An optimization algorithm that iteratively adjusts model parameters by moving in the direction of steepest decrease of the loss function.
• Parameters — The internal variables — weights and biases — that a model learns from data.