ICML 2021spotlight3 citations
WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points
Albert No, Taeho Yoon, Kwon Sehyun, Ernest K Ryu
Abstract
Generative adversarial networks (GAN) are a widely used class of deep generative models, but their minimax training dynamics are not understood very well. In this work, we show that GANs with a 2-layer infinite-width generator and a 2-layer finite-width discriminator trained with stochastic gradient ascent-descent have no spurious stationary points. We then show that when the width of the generator is finite but wide, there are no spurious stationary points within a ball whose radius becomes arbitrarily large (to cover the entire parameter space) as the width goes to infinity.
BibTeX
@InProceedings{pmlr-v139-no21a,
title = {WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points},
author = {No, Albert and Yoon, Taeho and Sehyun, Kwon and Ryu, Ernest K},
booktitle = {Proceedings of the 38th International Conference on Machine Learning},
pages = {8205--8215},
year = {2021},
editor = {Meila, Marina and Zhang, Tong},
volume = {139},
series = {Proceedings of Machine Learning Research},
month = {18--24 Jul},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v139/no21a/no21a.pdf},
url = {https://proceedings.mlr.press/v139/no21a.html},
abstract = {Generative adversarial networks (GAN) are a widely used class of deep generative models, but their minimax training dynamics are not understood very well. In this work, we show that GANs with a 2-layer infinite-width generator and a 2-layer finite-width discriminator trained with stochastic gradient ascent-descent have no spurious stationary points. We then show that when the width of the generator is finite but wide, there are no spurious stationary points within a ball whose radius becomes arbitrarily large (to cover the entire parameter space) as the width goes to infinity.}
}