Roeland Nusselder

Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization

Optimization of Binarized Neural Networks (BNNs) currently relies on real-valued latent weights to accumulate small update steps. In this paper, we argue that these latent weights cannot be treated analogously to weights in real-valued networks. Instead their main role is to provide inertia during t…