Beginner · Neural Networks

Output Layer

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. The final layer of a neural network that produces the prediction.

Technical Definition

The final layer of a neural network that produces the prediction.

How it works

Its shape depends on the task: one neuron with sigmoid for binary classification, k neurons with softmax for k-way classification, vocabulary-size logits for language modeling. The choice of activation in the output layer is tied to the loss function used during training.

Related Concepts

• Softmax Function — A function that converts a vector of raw scores into a probability distribution where all values sum to one.
• Sigmoid — An S-shaped activation function that squashes any input into the range (0, 1).
• Layer — A group of neurons that perform the same kind of transformation in parallel.