Variational Network Quantization
Jan Achterhold, Jan Mathias Koehler, Anke Schmeink, Tim Genewein
Abstract
In this paper, the preparation of a neural network for pruning and few-bit quantization is formulated as a variational inference problem. To this end, a quantizing prior that leads to a multi-modal, sparse posterior distribution over weights, is introduced and a differentiable Kullback-Leibler divergence approximation for this prior is derived. After training with Variational Network Quantization, weights can be replaced by deterministic quantization values with small to negligible loss of task accuracy (including pruning by setting weights to 0). The method does not require fine-tuning after quantization. Results are shown for ternary quantization on LeNet-5 (MNIST) and DenseNet (CIFAR-10).
BibTeX
@inproceedings{
achterhold2018variational,
title={Variational Network Quantization},
author={Jan Achterhold and Jan Mathias Koehler and Anke Schmeink and Tim Genewein},
booktitle={International Conference on Learning Representations},
year={2018},
url={https://openreview.net/forum?id=ry-TW-WAb},
}