Can Latent Alignments Improve Autoregressive Machine Translation?

Adi Haviv; Lior Vassertail; Omer Levy

NAACL 2021long7 citations

Can Latent Alignments Improve Autoregressive Machine Translation?

Abstract

Latent alignment objectives such as CTC and AXE significantly improve non-autoregressive machine translation models. Can they improve autoregressive models as well? We explore the possibility of training autoregressive machine translation models with latent alignment objectives, and observe that, in practice, this approach results in degenerate models. We provide a theoretical explanation for these empirical results, and prove that latent alignment objectives are incompatible with teacher forcing.

BibTeX

@inproceedings{haviv-etal-2021-latent,
    title = "Can Latent Alignments Improve Autoregressive Machine Translation?",
    author = "Haviv, Adi  and
      Vassertail, Lior  and
      Levy, Omer",
    editor = "Toutanova, Kristina  and
      Rumshisky, Anna  and
      Zettlemoyer, Luke  and
      Hakkani-Tur, Dilek  and
      Beltagy, Iz  and
      Bethard, Steven  and
      Cotterell, Ryan  and
      Chakraborty, Tanmoy  and
      Zhou, Yichao",
    booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
    month = jun,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.naacl-main.209/",
    doi = "10.18653/v1/2021.naacl-main.209",
    pages = "2637--2641"
}