Advanced · Safety
Counterfactual fairness
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. Counterfactual fairness ensures a model's output is consistent across individuals identical except for sensitive attributes.
Technical Definition
Counterfactual fairness ensures a model's output is consistent across individuals identical except for sensitive attributes.
How it works
Counterfactual fairness is a metric for evaluating model bias, focusing on whether an individual's outcome would change if certain sensitive attributes (like race or gender) were different, while all other factors remain the same. This helps identify if a model unfairly discriminates based on these attributes. Evaluating this is a key step in responsible AI development.
Related Concepts
- Fairness — The principle that AI systems should treat individuals and groups equitably.
- Bias (ethics/fairness) — Unfair prejudice or favoritism towards certain groups or things, which can influence data, system design, and user interactions.
- Responsible AI (RAI) — A holistic framework encompassing the ethical, legal, and societal implications of AI, promoting trustworthy and beneficial systems.