Intermediate · Safety

Red Teaming

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Systematically probing an AI system for failures, exploits, and harmful behavior before deployment.

Technical Definition

Systematically probing an AI system for failures, exploits, and harmful behavior before deployment.

How it works

Borrowed from cybersecurity, AI red teams attempt jailbreaks, elicit hallucinations, find data leaks, and stress-test edge cases. Frontier labs maintain dedicated red teams; regulators increasingly mandate red teaming for high-risk systems.

Related Concepts

• Safety — Designing AI systems to behave reliably and avoid causing harm under realistic conditions.
• Guardrails — Runtime checks that constrain model inputs and outputs to stay within safe, useful bounds.
• Adversarial Attack — An input crafted to fool a model into making a wrong prediction.