Adversarial Word Dilution as Text Data Augmentation in Low-Resource Regime
Junfan Chen, Richong Zhang, Zheyan Luo, Chunming Hu, Yongyi Mao
Abstract
Data augmentation is widely used in text classification, especially in the low-resource regime where a few examples for each class are available during training. Despite the success, generating data augmentations as hard positive examples that may increase their effectiveness is under-explored. This paper proposes an Adversarial Word Dilution (AWD) method that can generate hard positive examples as text data augmentations to train the low-resource text classification model efficiently. Our idea of augmenting the text data is to dilute the embedding of strong positive words by weighted mixing with unknown-word embedding, making the augmented inputs hard to be recognized as positive by the classification model. We adversarially learn the dilution weights through a constrained min-max optimization process with the guidance of the labels. Empirical studies on three benchmark datasets show that AWD can generate more effective data augmentations and outperform the state-of-the-art text data augmentation methods. The additional analysis demonstrates that the data augmentations generated by AWD are interpretable and can flexibly extend to new examples without further training.
BibTeX
@article{Chen_Zhang_Luo_Hu_Mao_2023, title={Adversarial Word Dilution as Text Data Augmentation in Low-Resource Regime}, volume={37}, url={https://ojs.aaai.org/index.php/AAAI/article/view/26486}, DOI={10.1609/aaai.v37i11.26486}, abstractNote={Data augmentation is widely used in text classification, especially in the low-resource regime where a few examples for each class are available during training. Despite the success, generating data augmentations as hard positive examples that may increase their effectiveness is under-explored. This paper proposes an Adversarial Word Dilution (AWD) method that can generate hard positive examples as text data augmentations to train the low-resource text classification model efficiently. Our idea of augmenting the text data is to dilute the embedding of strong positive words by weighted mixing with unknown-word embedding, making the augmented inputs hard to be recognized as positive by the classification model. We adversarially learn the dilution weights through a constrained min-max optimization process with the guidance of the labels. Empirical studies on three benchmark datasets show that AWD can generate more effective data augmentations and outperform the state-of-the-art text data augmentation methods. The additional analysis demonstrates that the data augmentations generated by AWD are interpretable and can flexibly extend to new examples without further training.}, number={11}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Chen, Junfan and Zhang, Richong and Luo, Zheyan and Hu, Chunming and Mao, Yongyi}, year={2023}, month={Jun.}, pages={12626-12634} }