Advanced · Research

JEPA (Joint Embedding Predictive Architecture)

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Yann LeCun's self-supervised architecture that predicts abstract representations of masked regions instead of reconstructing raw pixels or tokens.

Technical Definition

Yann LeCun's self-supervised architecture that predicts abstract representations of masked regions instead of reconstructing raw pixels or tokens.

How it works

JEPA learns by predicting the embedding of a target view from the embedding of a context view, conditioned on a positional latent. Because predictions happen in latent space rather than pixel space, the model is not penalized for unpredictable high-frequency detail and learns more semantic features. I-JEPA (images) and V-JEPA (video) are LeCun's roadmap toward world models that learn physics from observation.

Related Concepts

• Contrastive Learning — A self-supervised technique that learns representations by pulling similar samples together and pushing dissimilar ones apart.
• Self-Supervised Learning — A training paradigm that generates supervisory signals from the data itself, eliminating the need for human labels.
• World model — A neural network simulating real-world dynamics, including physics, to generate realistic environments.
• Representation Learning — The process of automatically discovering meaningful representations of data from raw inputs.