Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize

Youngsuk Park; Sauptik Dhar; Stephen P. Boyd; Mohak Shah

ICASSP 2020accepted0 citations

Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize

Youngsuk Park, Sauptik Dhar, Stephen P. Boyd, Mohak Shah

Abstract

This paper proposes an adaptive metric selection strategy called diagonal Barzilai-Borwein (DBB) stepsize for the popular Variable Metric Proximal Gradient (VM-PG) algorithm [1], [2]. The proposed approach better captures the local geometry of the problem while keeping the per-step computation cost similar to the widely used scalar Barzilai-Borwein (BB) stepsize. We provide the theoretical convergence analysis for VM-PG using DBB stepsize. Finally, our empirical results show ~10 - 40 % improvement in convergence times for the VM-PG using DBB compared to the BB stepsize for different machine learning problems on several datasets.

BibTeX

@inproceedings{icassp2020_variablemetricpr,
  title = {Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize},
  author = {Youngsuk Park and Sauptik Dhar and Stephen P. Boyd and Mohak Shah},
  booktitle = {ICASSP 2020},
  year = {2020}
}