Viktor Yanush

Path Sample-Analytic Gradient Estimators for Stochastic Binary Networks

In neural networks with binary activations and or binary weights the training by gradient descent is complicated as the model has piecewise constant response. We consider stochastic binary networks, obtained by adding noises in front of activations. The expected model response becomes a smooth funct…