Pre-trained Language Model Based Active Learning for Sentence Matching

Guirong Bai; Shizhu He; Kang Liu; Jun Zhao; Zaiqing Nie

COLING 2020main10 citations

Pre-trained Language Model Based Active Learning for Sentence Matching

Guirong Bai, Shizhu He, Kang Liu, Jun Zhao, Zaiqing Nie

Abstract

Active learning is able to significantly reduce the annotation cost for data-driven techniques. However, previous active learning approaches for natural language processing mainly depend on the entropy-based uncertainty criterion, and ignore the characteristics of natural language. In this paper, we propose a pre-trained language model based active learning approach for sentence matching. Differing from previous active learning, it can provide linguistic criteria from the pre-trained language model to measure instances and help select more effective instances for annotation. Experiments demonstrate our approach can achieve greater accuracy with fewer labeled training instances.

BibTeX

@inproceedings{bai-etal-2020-pre,
    title = "Pre-trained Language Model Based Active Learning for Sentence Matching",
    author = "Bai, Guirong  and
      He, Shizhu  and
      Liu, Kang  and
      Zhao, Jun  and
      Nie, Zaiqing",
    editor = "Scott, Donia  and
      Bel, Nuria  and
      Zong, Chengqing",
    booktitle = "Proceedings of the 28th International Conference on Computational Linguistics",
    month = dec,
    year = "2020",
    address = "Barcelona, Spain (Online)",
    publisher = "International Committee on Computational Linguistics",
    url = "https://aclanthology.org/2020.coling-main.130/",
    doi = "10.18653/v1/2020.coling-main.130",
    pages = "1495--1504"
}