Advanced · Generative AI

Variational Autoencoder (VAE)

Visual diagram · (in preparation) · Math · Worked example · 3 difficulty levels.

TL;DR. A generative model that learns a smooth, probabilistic latent space enabling meaningful interpolation and data generation.

Technical Definition

A generative model that learns a smooth, probabilistic latent space enabling meaningful interpolation and data generation.

How it works

VAEs make the latent space probabilistic. The encoder outputs mean and variance, the reparameterization trick enables backpropagation through sampling. The loss combines reconstruction error with KL divergence to keep the latent distribution smooth.

Mathematical Notation

L = E[log p(x|z)] − KL(q(z|x) ‖ p(z))

Balances reconstruction quality and latent regularization (how close q(z|x) is to the prior).

Related Concepts

• Embedding — A dense vector representation that captures semantic meaning, mapping discrete items like words into continuous mathematical space.
• Generative Adversarial Network (GAN) — Two neural networks — a generator and discriminator — compete against each other to produce increasingly realistic synthetic data.
• Autoencoder — A neural network that learns compressed representations by training to reconstruct its own input through a bottleneck layer.