ICLR 2019poster104 citations
On the loss landscape of a class of deep neural networks with no bad local valleys
Quynh Nguyen, Mahesh Chandra Mukkamala, Matthias Hein
Abstract
We identify a class of over-parameterized deep neural networks with standard activation functions and cross-entropy loss which provably have no bad local valley, in the sense that from any point in parameter space there exists a continuous path on which the cross-entropy loss is non-increasing and gets arbitrarily close to zero. This implies that these networks have no sub-optimal strict local minima.
loss landscapelocal minimadeep neural networks
BibTeX
@inproceedings{
nguyen2018on,
title={On the loss landscape of a class of deep neural networks with no bad local valleys},
author={Quynh Nguyen and Mahesh Chandra Mukkamala and Matthias Hein},
booktitle={International Conference on Learning Representations},
year={2019},
url={https://openreview.net/forum?id=HJgXsjA5tQ},
}