Beginner · Neural Networks

Bias (model parameter)

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. A learnable constant added to a neuron's weighted input that shifts the activation function.

Technical Definition

A learnable constant added to a neuron's weighted input that shifts the activation function.

How it works

For a neuron computing y = f(Σwᵢxᵢ + b), the bias b lets the activation fire even when all inputs are zero, giving the network flexibility to fit data that doesn't pass through the origin. Not to be confused with statistical or societal bias.

Related Concepts

• Neural Network — A computing system inspired by biological neural networks that learns patterns from data through interconnected layers of nodes.
• Activation Function — A non-linear function applied to a neuron's output, enabling the network to learn complex, non-linear relationships.
• Weights — The learnable numerical parameters that determine how a neural network transforms its inputs.