AIConfPaper
← Search

Ganlin Song

2 accepted papers

2021

Convergence and Alignment of Gradient Descent with Random Backpropagation Weights

Ganlin Song, Ruitu Xu, John Lafferty
NeurIPS 2021poster

Stochastic gradient descent with backpropagation is the workhorse of artificial neural networks. It has long been recognized that backpropagation fails to be a biologically plausible algorithm. Fundamentally, it is a non-local procedure---updating one neuron's synaptic weights requires knowledge of…

Cited by 24Source ↗PDF ↗Scholar ↗
2019

Surfing: Iterative Optimization Over Incrementally Trained Deep Networks

Ganlin Song, Zhou Fan, John Lafferty
NeurIPS 2019spotlight

We investigate a sequential optimization procedure to minimize the empirical risk functional $f_{\hat\theta}(x) = \frac{1}{2}\|G_{\hat\theta}(x) - y\|^2$ for certain families of deep networks $G_{\theta}(x)$. The approach is to optimize a sequence of objective functions that use network parameters o…

Cited by 24Source ↗PDF ↗code ↗Scholar ↗