NAACL 2021long10 citations
Targeted Adversarial Training for Natural Language Understanding
Lis Pereira, Xiaodong Liu, Hao Cheng, Hoifung Poon, Jianfeng Gao, Ichiro Kobayashi
Abstract
We present a simple yet effective Targeted Adversarial Training (TAT) algorithm to improve adversarial training for natural language understanding. The key idea is to introspect current mistakes and prioritize adversarial training steps to where the model errs the most. Experiments show that TAT can significantly improve accuracy over standard adversarial training on GLUE and attain new state-of-the-art zero-shot results on XNLI. Our code will be released upon acceptance of the paper.
BibTeX
@inproceedings{pereira-etal-2021-targeted,
title = "Targeted Adversarial Training for Natural Language Understanding",
author = "Pereira, Lis and
Liu, Xiaodong and
Cheng, Hao and
Poon, Hoifung and
Gao, Jianfeng and
Kobayashi, Ichiro",
editor = "Toutanova, Kristina and
Rumshisky, Anna and
Zettlemoyer, Luke and
Hakkani-Tur, Dilek and
Beltagy, Iz and
Bethard, Steven and
Cotterell, Ryan and
Chakraborty, Tanmoy and
Zhou, Yichao",
booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
month = jun,
year = "2021",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.naacl-main.424/",
doi = "10.18653/v1/2021.naacl-main.424",
pages = "5385--5393"
}