Word-level Perturbation Considering Word Length and Compositional Subwords
Tatsuya Hiraoka, Sho Takase, Kei Uchiumi, Atsushi Keyaki, Naoaki Okazaki
Abstract
We present two simple modifications for word-level perturbation: Word Replacement considering Length (WR-L) and Compositional Word Replacement (CWR).In conventional word replacement, a word in an input is replaced with a word sampled from the entire vocabulary, regardless of the length and context of the target word.WR-L considers the length of a target word by sampling words from the Poisson distribution.CWR considers the compositional candidates by restricting the source of sampling to related words that appear in subword regularization. Experimental results showed that the combination of WR-L and CWR improved the performance of text classification and machine translation.
BibTeX
@inproceedings{hiraoka-etal-2022-word,
title = "Word-level Perturbation Considering Word Length and Compositional Subwords",
author = "Hiraoka, Tatsuya and
Takase, Sho and
Uchiumi, Kei and
Keyaki, Atsushi and
Okazaki, Naoaki",
editor = "Muresan, Smaranda and
Nakov, Preslav and
Villavicencio, Aline",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2022",
month = may,
year = "2022",
address = "Dublin, Ireland",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.findings-acl.258/",
doi = "10.18653/v1/2022.findings-acl.258",
pages = "3268--3275"
}