EMNLP 2021finding5 citations
Bag of Tricks for Optimizing Transformer Efficiency
Ye Lin, Yanyang Li, Tong Xiao, Jingbo Zhu
Abstract
Improving Transformer efficiency has become increasingly attractive recently. A wide range of methods has been proposed, e.g., pruning, quantization, new architectures and etc. But these methods are either sophisticated in implementation or dependent on hardware. In this paper, we show that the efficiency of Transformer can be improved by combining some simple and hardware-agnostic methods, including tuning hyper-parameters, better design choices and training strategies. On the WMT news translation tasks, we improve the inference efficiency of a strong Transformer system by 3.80x on CPU and 2.52x on GPU.
BibTeX
@inproceedings{lin-etal-2021-bag-tricks,
title = "Bag of Tricks for Optimizing Transformer Efficiency",
author = "Lin, Ye and
Li, Yanyang and
Xiao, Tong and
Zhu, Jingbo",
editor = "Moens, Marie-Francine and
Huang, Xuanjing and
Specia, Lucia and
Yih, Scott Wen-tau",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
month = nov,
year = "2021",
address = "Punta Cana, Dominican Republic",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.findings-emnlp.357/",
doi = "10.18653/v1/2021.findings-emnlp.357",
pages = "4227--4233"
}