Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution

Xavier Garcia; Noah Constant; Ankur Parikh; Orhan Firat

NAACL 2021long44 citations

Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution

Xavier Garcia, Noah Constant, Ankur Parikh, Orhan Firat

Abstract

We propose a straightforward vocabulary adaptation scheme to extend the language capacity of multilingual machine translation models, paving the way towards efficient continual learning for multilingual machine translation. Our approach is suitable for large-scale datasets, applies to distant languages with unseen scripts, incurs only minor degradation on the translation performance for the original language pairs and provides competitive performance even in the case where we only possess monolingual data for the new languages.

BibTeX

@inproceedings{garcia-etal-2021-towards,
    title = "Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution",
    author = "Garcia, Xavier  and
      Constant, Noah  and
      Parikh, Ankur  and
      Firat, Orhan",
    editor = "Toutanova, Kristina  and
      Rumshisky, Anna  and
      Zettlemoyer, Luke  and
      Hakkani-Tur, Dilek  and
      Beltagy, Iz  and
      Bethard, Steven  and
      Cotterell, Ryan  and
      Chakraborty, Tanmoy  and
      Zhou, Yichao",
    booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
    month = jun,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.naacl-main.93/",
    doi = "10.18653/v1/2021.naacl-main.93",
    pages = "1184--1192"
}