Matteo Spallanzani

Training Quantised Neural Networks With STE Variants: The Additive Noise Annealing Algorithm

Training quantised neural networks (QNNs) is a non-differentiable optimisation problem since weights and features are output by piecewise constant functions. The standard solution is to apply the straight-through estimator (STE), using different functions during the inference and gradient computatio…