Written Justifications are Key to Aggregate Crowdsourced Forecasts

Saketh Kotamraju; Eduardo Blanco

EMNLP 2021finding1 citations

Written Justifications are Key to Aggregate Crowdsourced Forecasts

Abstract

This paper demonstrates that aggregating crowdsourced forecasts benefits from modeling the written justifications provided by forecasters. Our experiments show that the majority and weighted vote baselines are competitive, and that the written justifications are beneficial to call a question throughout its life except in the last quarter. We also conduct an error analysis shedding light into the characteristics that make a justification unreliable.

BibTeX

@inproceedings{kotamraju-blanco-2021-written-justifications,
    title = "Written Justifications are Key to Aggregate Crowdsourced Forecasts",
    author = "Kotamraju, Saketh  and
      Blanco, Eduardo",
    editor = "Moens, Marie-Francine  and
      Huang, Xuanjing  and
      Specia, Lucia  and
      Yih, Scott Wen-tau",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
    month = nov,
    year = "2021",
    address = "Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.findings-emnlp.355/",
    doi = "10.18653/v1/2021.findings-emnlp.355",
    pages = "4206--4216"
}