Advanced · Research

AI Treadmill

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. The continuous cycle of increasing AI model size, complexity, and resource demands to achieve incremental performance gains.

Technical Definition

The continuous cycle of increasing AI model size, complexity, and resource demands to achieve incremental performance gains.

How it works

The AI treadmill describes the ever-escalating requirements for training cutting-edge AI models, driven by the belief that larger models simply perform better. This leads to substantial computational costs, energy consumption, and environmental impact. Researchers are exploring alternatives to break this cycle, such as parameter-efficient methods and new architectures that achieve better performance with fewer resources.

Related Concepts

• Scaling Laws — Empirical power-law relationships predicting how LLM loss decreases as model size, dataset size, and compute increase together.