Domain Generalisation of NMT: Fusing Adapters with Leave-One-Domain-Out Training

Thuy-Trang Vu; Shahram Khadivi; Dinh Phung; Gholamreza Haffari

ACL 2022findings12 citations

Domain Generalisation of NMT: Fusing Adapters with Leave-One-Domain-Out Training

Thuy-Trang Vu, Shahram Khadivi, Dinh Phung, Gholamreza Haffari

Abstract

Generalising to unseen domains is under-explored and remains a challenge in neural machine translation. Inspired by recent research in parameter-efficient transfer learning from pretrained models, this paper proposes a fusion-based generalisation method that learns to combine domain-specific parameters. We propose a leave-one-domain-out training strategy to avoid information leaking to address the challenge of not knowing the test domain during training time. Empirical results on three language pairs show that our proposed fusion method outperforms other baselines up to +0.8 BLEU score on average.

BibTeX

@inproceedings{vu-etal-2022-domain,
    title = "Domain Generalisation of {NMT}: Fusing Adapters with Leave-One-Domain-Out Training",
    author = "Vu, Thuy-Trang  and
      Khadivi, Shahram  and
      Phung, Dinh  and
      Haffari, Gholamreza",
    editor = "Muresan, Smaranda  and
      Nakov, Preslav  and
      Villavicencio, Aline",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2022",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.findings-acl.49/",
    doi = "10.18653/v1/2022.findings-acl.49",
    pages = "582--588"
}