Advanced · Neural Networks

ResNet (Residual Network)

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. A type of convolutional neural network that uses 'skip connections' to allow for training much deeper models.

Technical Definition

A type of convolutional neural network that uses 'skip connections' to allow for training much deeper models.

How it works

ResNets address the degradation problem in training very deep neural networks, where adding more layers can lead to worse performance. They introduce skip connections (or residual connections) that bypass one or more layers, allowing gradients to flow directly through the network. This enables the training of networks with hundreds or even thousands of layers, leading to significant advances in image recognition and computer vision tasks.