Intermediate · Safety

Jailbreak

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Adversarial prompts that bypass an aligned model's safety training, eliciting outputs the model would normally refuse.

Technical Definition

Adversarial prompts that bypass an aligned model's safety training, eliciting outputs the model would normally refuse.

How it works

Jailbreaks range from social-engineering personas (DAN, grandma exploits) to gradient-based suffix attacks (GCG) and many-shot in-context attacks that flood the context with rule-breaking examples. Defenses include adversarial training, output filters, and constitutional methods, but no current model is fully robust. Tracking jailbreaks is a core activity for red teams and AI safety labs.

Related Concepts

• AI Safety & Alignment — The field ensuring AI systems behave as intended, remain under human control, and avoid unintended harm.
• Red Teaming — Systematically probing an AI system for failures, exploits, and harmful behavior before deployment.
• Prompt Injection — An attack where untrusted input (a webpage, email, tool output) contains instructions that hijack an LLM's behavior in an agentic context.