Beginner · Neural Networks

Tanh

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. An activation function that maps inputs to the range (-1, 1), centered at zero.

Technical Definition

An activation function that maps inputs to the range (-1, 1), centered at zero.

How it works

tanh(x) = (eˣ - e⁻ˣ) / (eˣ + e⁻ˣ). Zero-centered outputs make it slightly better behaved than sigmoid for hidden layers, but it still suffers from vanishing gradients at the extremes. Common in older RNNs and gating components.

Related Concepts

• Activation Function — A non-linear function applied to a neuron's output, enabling the network to learn complex, non-linear relationships.
• Recurrent Neural Network (RNN) — A neural network with loops that maintain hidden state, designed to process sequential data like text and time series.
• Sigmoid — An S-shaped activation function that squashes any input into the range (0, 1).