Intermediate · Neural Networks

Autoencoder

Visual diagram · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. A neural network that learns compressed representations by training to reconstruct its own input through a bottleneck layer.

Technical Definition

A neural network that learns compressed representations by training to reconstruct its own input through a bottleneck layer.

How it works

Autoencoders consist of an encoder that compresses input into a lower-dimensional latent representation and a decoder that reconstructs the original. The bottleneck forces the network to learn the most important features. Variants include VAEs (probabilistic), Denoising (learns to remove noise), and Sparse (enforces sparsity).

Visual Explanation (layers)

Input (784 dims) → Encoder → Latent Space (32 dims) → Decoder → Reconstructed Input (784 dims)

Related Concepts

• Neural Network — A computing system inspired by biological neural networks that learns patterns from data through interconnected layers of nodes.
• Embedding — A dense vector representation that captures semantic meaning, mapping discrete items like words into continuous mathematical space.
• Deep Learning — A subset of machine learning using neural networks with many layers to learn hierarchical representations from large datasets.