Beginner · Neural Networks

Sigmoid

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. An S-shaped activation function that squashes any input into the range (0, 1).

Technical Definition

An S-shaped activation function that squashes any input into the range (0, 1).

How it works

σ(x) = 1 / (1 + e^-x). Useful for binary classification outputs and gating mechanisms in LSTMs/GRUs. Rarely used in deep hidden layers today because it saturates for large |x|, causing vanishing gradients.

Related Concepts

• Activation Function — A non-linear function applied to a neuron's output, enabling the network to learn complex, non-linear relationships.
• Softmax Function — A function that converts a vector of raw scores into a probability distribution where all values sum to one.
• Tanh — An activation function that maps inputs to the range (-1, 1), centered at zero.