← Search

Steven Chu Hong Hoi

2 accepted papers

2020

Theory-Inspired Path-Regularized Differential Network Architecture Search

NeurIPS 2020oral

Despite its high search efficiency, differential architecture search (DARTS) often selects network architectures with dominated skip connections which lead to performance degradation. However, theoretical understandings on this issue remain absent, hindering the development of more advanced methods…

Cited by 72SourcePDFScholar
2020

Towards Theoretically Understanding Why Sgd Generalizes Better Than Adam in Deep Learning

NeurIPS 2020poster

It is not clear yet why ADAM-alike adaptive gradient algorithms suffer from worse generalization performance than SGD despite their faster training speed. This work aims to provide understandings on this generalization gap by analyzing their local convergence behaviors. Specifically, we observe…

Cited by 355SourcePDFScholar