Intermediate · NLP

BLEU Score

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. A metric that measures how close machine-generated text is to human reference translations.

Technical Definition

A metric that measures how close machine-generated text is to human reference translations.

How it works

BLEU compares n-gram overlap between candidate and reference text, with a brevity penalty for short outputs. Range 0–1 (or 0–100). Long the standard for machine translation, it is increasingly criticized for missing semantic equivalence — modern evaluation often pairs it with embedding-based metrics or LLM-as-judge.

Related Concepts

• Perplexity — A metric measuring how well a language model predicts text — lower means less 'surprised' by the data.
• ROUGE Score — A family of metrics that measure overlap between generated and reference text, used mainly for summarization.