NeurIPS 2020poster102 citations
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot
Jingtong Su, Yihang Chen, Tianle Cai, Tianhao Wu, Ruiqi Gao, Liwei Wang, Jason Lee
Abstract
Network pruning is a method for reducing test-time computational resource requirements with minimal performance degradation. Conventional wisdom of pruning algorithms suggests that: (1) Pruning methods exploit information from training data to find good subnetworks; (2) The architecture of the pruned network is crucial for good performance. In this paper, we conduct sanity checks for the above beliefs on several recent unstructured pruning methods and surprisingly find that: (1) A set of methods which aims to find good subnetworks of the randomly-initialized network (which we call
BibTeX
@inproceedings{NEURIPS2020_eae27d77,
author = {Su, Jingtong and Chen, Yihang and Cai, Tianle and Wu, Tianhao and Gao, Ruiqi and Wang, Liwei and Lee, Jason D},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Larochelle and M. Ranzato and R. Hadsell and M.F. Balcan and H. Lin},
pages = {20390--20401},
publisher = {Curran Associates, Inc.},
title = {Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot},
url = {https://proceedings.neurips.cc/paper_files/paper/2020/file/eae27d77ca20db309e056e3d2dcd7d69-Paper.pdf},
volume = {33},
year = {2020}
}