Jonathan Ivey

NUTMEG: Separating Signal From Noise in Annotator Disagreement

NLP models often rely on human-labeled data for training and evaluation. Many approaches crowdsource this data from a large number of annotators with varying skills, backgrounds, and motivations, resulting in conflicting annotations. These conflicts have traditionally been resolved by aggregation me