Meric Altug Gemalmaz

Accounting for Confirmation Bias in Crowdsourced Label Aggregation

Collecting large-scale human-annotated datasets via crowdsourcing to train and improve automated models is a prominent human-in-the-loop approach to integrate human and machine intelligence. However, together with their unique intelligence, humans also come with their biases and subjective beliefs,…