Optimal Crowdsourced Classification with a Reject Option in the Presence of Spammers

Qunwei Li; Pramod K. Varshney

ICASSP 2018accepted0 citations

Optimal Crowdsourced Classification with a Reject Option in the Presence of Spammers

Abstract

We explore the design of an effective crowdsourcing system for an M -ary classification task. Crowd workers complete simple binary microtasks whose results are aggregated to give the final decision. We consider the scenario where the workers have a reject option so that they are allowed to skip microtasks when they are unable to or choose not to respond to binary microtasks. We present an aggregation approach using a weighted majority voting rule, where each worker's response is assigned an optimized weight to maximize crowd's classification performance.

BibTeX

@inproceedings{icassp2018_optimalcrowdsour,
  title = {Optimal Crowdsourced Classification with a Reject Option in the Presence of Spammers},
  author = {Qunwei Li and Pramod K. Varshney},
  booktitle = {ICASSP 2018},
  year = {2018}
}