Intermediate · Evaluation
Human evaluation
Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.
TL;DR. Assessing model performance by using human judgment to rate its outputs.
Technical Definition
Assessing model performance by using human judgment to rate its outputs.
How it works
Human evaluation involves real people assessing the quality of an AI model's output, which is especially useful for tasks where there isn't a single correct answer, like translation or creative text generation. For example, bilingual speakers might evaluate the fluency and accuracy of a machine translation. This complements automated metrics by capturing nuances that algorithms might miss.
Related Concepts
- Evaluation — The process of measuring a model's quality or comparing different models.
- Generative AI — An AI field focused on creating models that can generate novel and complex content, such as text, images, audio, and video, that is coherent and original.