Advanced · Safety

Adversarial Attack

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. An input crafted to fool a model into making a wrong prediction.

Technical Definition

An input crafted to fool a model into making a wrong prediction.

How it works

Adversarial examples include imperceptibly perturbed images that flip image classifiers, jailbreak prompts that bypass LLM safety, and data poisoning that backdoors models at training time. Defenses range from adversarial training to robust aggregation and runtime detection.

Related Concepts

• AI Safety & Alignment — The field ensuring AI systems behave as intended, remain under human control, and avoid unintended harm.
• Guardrails — Runtime checks that constrain model inputs and outputs to stay within safe, useful bounds.
• Red Teaming — Systematically probing an AI system for failures, exploits, and harmful behavior before deployment.