← Search

Y. Jennifer Sun

5 accepted papers

2023

Sketchy: Memory-efficient Adaptive Regularization with Frequent Directions

NeurIPS 2023poster

Adaptive regularization methods that exploit more than the diagonal entries exhibit state of the art performance for many tasks, but can be prohibitive in terms of memory and running time. We find the spectra of the Kronecker-factored gradient covariance matrix in deep learning (DL) training tasks a…

Cited by 14SourcePDFScholar