Intermediate · NLP

GRU

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. A gated recurrent unit — a streamlined alternative to the LSTM with fewer parameters.

Technical Definition

A gated recurrent unit — a streamlined alternative to the LSTM with fewer parameters.

How it works

GRUs combine the forget and input gates of an LSTM into a single update gate, and merge the cell state with the hidden state. They train faster and often match LSTM performance on shorter sequences, though both have been largely superseded by Transformers for most language tasks.

Related Concepts

• Recurrent Neural Network (RNN) — A neural network with loops that maintain hidden state, designed to process sequential data like text and time series.
• Long Short-Term Memory (LSTM) — An RNN variant with gating mechanisms that can learn long-range dependencies without suffering from vanishing gradients.