Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation

Samuel Kiegeland; Julia Kreutzer

NAACL 2021long52 citations

Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation

Abstract

Policy gradient algorithms have found wide adoption in NLP, but have recently become subject to criticism, doubting their suitability for NMT. Choshen et al. (2020) identify multiple weaknesses and suspect that their success is determined by the shape of output distributions rather than the reward. In this paper, we revisit these claims and study them under a wider range of configurations. Our experiments on in-domain and cross-domain adaptation reveal the importance of exploration and reward scaling, and provide empirical counter-evidence to these claims.

BibTeX

@inproceedings{kiegeland-kreutzer-2021-revisiting,
    title = "Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation",
    author = "Kiegeland, Samuel  and
      Kreutzer, Julia",
    editor = "Toutanova, Kristina  and
      Rumshisky, Anna  and
      Zettlemoyer, Luke  and
      Hakkani-Tur, Dilek  and
      Beltagy, Iz  and
      Bethard, Steven  and
      Cotterell, Ryan  and
      Chakraborty, Tanmoy  and
      Zhou, Yichao",
    booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
    month = jun,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.naacl-main.133/",
    doi = "10.18653/v1/2021.naacl-main.133",
    pages = "1673--1681"
}