Home › Glossary › Generative AI › Diffusion Transformer (DiT)

Advanced · Generative AI

Diffusion Transformer (DiT)

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. A diffusion model that replaces the U-Net denoiser with a Transformer operating on image or video patches, scaling cleanly with compute.

Technical Definition

A diffusion model that replaces the U-Net denoiser with a Transformer operating on image or video patches, scaling cleanly with compute.

How it works

DiT (Peebles & Xie, 2022) tokenizes the noisy latent into patches, adds positional and timestep embeddings, and processes them with a standard Transformer that predicts noise. Performance follows clean scaling laws in depth and width, and the architecture underpins Stable Diffusion 3, Sora, and Flux. Conditioning is injected via adaptive LayerNorm (adaLN-Zero).

Related Concepts

  • Transformer — An architecture that uses self-attention to process sequences in parallel, powering modern language models like GPT and BERT.
  • Diffusion Model — A generative model that learns to create data by reversing a gradual noising process, producing high-quality images and audio.
  • Sora — OpenAI's text-to-video diffusion-transformer model that generates minute-long high-resolution clips with strong temporal coherence and 3D consistency.