Intermediate · Evaluation
Fairness metric
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. A quantifiable measure used to assess the fairness of a model's predictions across different groups.
Technical Definition
A quantifiable measure used to assess the fairness of a model's predictions across different groups.
How it works
A fairness metric is a mathematical definition that allows for the measurement of fairness in machine learning models across various sensitive attributes. Common examples include equalized odds, predictive parity, and demographic parity, though achieving all simultaneously can be impossible.
Related Concepts
- Bias (ethics/fairness) — Unfair prejudice or favoritism towards certain groups or things, which can influence data, system design, and user interactions.
- Demographic parity — A fairness metric ensuring prediction rates are the same across different demographic groups.
- Equalized odds — A fairness metric where true positive rate and false negative rate are equal across all groups.
- Responsible AI (RAI) — A holistic framework encompassing the ethical, legal, and societal implications of AI, promoting trustworthy and beneficial systems.