Adrian Cheung

Inconsistencies in Crowdsourced Slot-Filling Annotations: A Typology and Identification Methods

Slot-filling models in task-driven dialog systems rely on carefully annotated training data. However, annotations by crowd workers are often inconsistent or contain errors. Simple solutions like manually checking annotations or having multiple workers label each sample are expensive and waste effort…