2020
Residual Distillation: Towards Portable Deep Neural Networks without Shortcuts
NeurIPS 2020poster
By transferring both features and gradients between different layers, shortcut connections explored by ResNets allow us to effectively train very deep neural networks up to hundreds of layers. However, the additional computation costs induced by those shortcuts are often overlooked. For example, du…