Ruijuan Chen

pbSGD: Powered Stochastic Gradient Descent Methods for Accelerated Non-Convex Optimization

We propose a novel technique for improving the stochastic gradient descent (SGD) method to train deep networks, which we term pbSGD. The proposed pbSGD method simply raises the stochastic gradient to a certain power elementwise during iterations and introduces only one additional parameter, namely,…