Beginner · Generative AI

Temperature (Sampling)

Visual diagram · (in preparation) · Math · Worked example · 3 difficulty levels.

TL;DR. A parameter controlling output randomness — lower values are more focused, higher values more creative.

Technical Definition

A parameter controlling output randomness — lower values are more focused, higher values more creative.

How it works

Temperature scales logits before softmax. T<1 sharpens the distribution (more deterministic). T>1 flattens it (more random). T→0 becomes greedy decoding. Other sampling strategies include top-k and top-p/nucleus sampling.

Mathematical Notation

P(xᵢ) = exp(zᵢ/T) / Σⱼ exp(zⱼ/T)

Dividing logits by T controls distribution sharpness. T=0.1 is near-deterministic, T=2.0 is highly random.

Related Concepts

• Large Language Model (LLM) — A massive neural network trained on vast text corpora to understand and generate human language with remarkable fluency.
• Softmax Function — A function that converts a vector of raw scores into a probability distribution where all values sum to one.
• Prompt Engineering — The art of crafting effective input instructions to guide LLM behavior without changing model weights.