Mengyun Chen — accepted AI-conference papers

2021

Enhance Curvature Information by Structured Stochastic Quasi-Newton Methods

Minghan Yang, Dong Xu, Hongyu Chen, Zaiwen Wen, Mengyun Chen

CVPR 2021poster

In this paper, we consider stochastic second-order methods for minimizing a finite summation of nonconvex functions. One important key is to find an ingenious but cheap scheme to incorporate local curvature information. Since the true Hessian matrix is often a combination of a cheap part and an expe…

Cited by 10 PDF ↗Scholar ↗

2021

THOR, Trace-based Hardware-driven Layer-Oriented Natural Gradient Descent Computation

Mengyun Chen, Kaixin Gao, Xiaolei Liu, Zidong Wang, Ningxi Ni, Qian Zhang +8

AAAI 2021technical

It is well-known that second-order optimizer can accelerate the training of deep neural networks, however, the huge computation cost of second-order optimization makes it impractical to apply in real practice. In order to reduce the cost, many methods have been proposed to approximate a second-order…

Cited by 9 Source ↗PDF ↗Scholar ↗