Intermediate · Research

Test-Time Compute

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Spending more inference compute (longer chains of thought, sampling, search) to improve answers, often outperforming a larger model.

Technical Definition

Spending more inference compute (longer chains of thought, sampling, search) to improve answers, often outperforming a larger model.

How it works

Popularized by OpenAI's o1 and DeepSeek-R1, test-time compute trades latency for accuracy by letting the model reason for thousands of tokens before answering, optionally combined with self-consistency, best-of-N, or process-reward-guided search. Snell et al. (2024) showed it can be Pareto-optimal versus parameter scaling on hard reasoning, reshaping how labs allocate budget.

Related Concepts

• Chain-of-Thought (CoT) Prompting — Asking models to show step-by-step reasoning before giving a final answer, improving accuracy on complex tasks.
• Self-Consistency — A decoding strategy that samples multiple chain-of-thought reasoning paths and returns the majority-voted answer for improved accuracy.
• Scaling Laws — Empirical power-law relationships predicting how LLM loss decreases as model size, dataset size, and compute increase together.