Intermediate · Safety

Model Alignment

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. The process of training AI models, especially LLMs, to behave in accordance with human values, intentions, and ethical principles.

Technical Definition

The process of training AI models, especially LLMs, to behave in accordance with human values, intentions, and ethical principles.

How it works

Model alignment is a crucial area of AI safety research, aiming to ensure that AI systems are beneficial and safe. Techniques like RLHF are employed to align models with human preferences, reducing harmful outputs and promoting helpful, honest, and harmless behavior. It addresses the challenge of getting AI to do what we intend, rather than just what we instruct literally.

Related Concepts

• RLHF (Reinforcement Learning from Human Feedback) — A technique that aligns LLM outputs with human preferences by training a reward model from human comparisons.
• AI Safety & Alignment — The field ensuring AI systems behave as intended, remain under human control, and avoid unintended harm.