ACL 2021short25 citations
Domain-Adaptive Pretraining Methods for Dialogue Understanding
Han Wu, Kun Xu, Linfeng Song, Lifeng Jin, Haisong Zhang, Linqi Song
Abstract
Language models like BERT and SpanBERT pretrained on open-domain data have obtained impressive gains on various NLP tasks. In this paper, we probe the effectiveness of domain-adaptive pretraining objectives on downstream tasks. In particular, three objectives, including a novel objective focusing on modeling predicate-argument relations, are evaluated on two challenging dialogue understanding tasks. Experimental results demonstrate that domain-adaptive pretraining with proper objectives can significantly improve the performance of a strong baseline on these tasks, achieving the new state-of-the-art performances.
BibTeX
@inproceedings{wu-etal-2021-domain,
title = "Domain-Adaptive Pretraining Methods for Dialogue Understanding",
author = "Wu, Han and
Xu, Kun and
Song, Linfeng and
Jin, Lifeng and
Zhang, Haisong and
Song, Linqi",
editor = "Zong, Chengqing and
Xia, Fei and
Li, Wenjie and
Navigli, Roberto",
booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)",
month = aug,
year = "2021",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.acl-short.84/",
doi = "10.18653/v1/2021.acl-short.84",
pages = "665--669"
}