Gabriel Singer

Optimal Fair Aggregation of Crowdsourced Noisy Labels using Demographic Parity Constraints

In many machine learning applications acquiring reliable ground-truth labels is costly, or unfeasible, leading practitioners to rely on crowdsourcing and aggregation of noisy human annotations. When labels are subjective, however, aggregation may amplify individual biases, particularly with respect …