Differentiable Transportation Pruning
Yunqiang Li, Jan C. van Gemert, Torsten Hoefler, Bert Moons, Evangelos Eleftheriou, Bram-Ernst Verhoef
Abstract
Deep learning algorithms are increasingly employed at the edge. However, edge devices are resource constrained and thus require efficient deployment of deep neural networks. Pruning methods are a key tool for edge deployment as they can improve storage, compute, memory bandwidth, and energy usage. In this paper we propose a novel accurate pruning technique that allows precise control over the output network size. Our method uses an efficient optimal transportation scheme which we make end-to-end differentiable and which automatically tunes the exploration-exploitation behavior of the algorithm to find accurate sparse sub-networks. We show that our method achieves state-of-the-art performance compared to previous pruning methods on 3 different datasets, using 5 different models, across a wide range of pruning ratios, and with two types of sparsity budgets and pruning granularities.
BibTeX
@inproceedings{iccv2023_differentiabletr,
title = {Differentiable Transportation Pruning},
author = {Yunqiang Li and Jan C. van Gemert and Torsten Hoefler and Bert Moons and Evangelos Eleftheriou and Bram-Ernst Verhoef},
booktitle = {ICCV 2023},
year = {2023}
}