Shengping Xie

FROM POWERSGD TO POWERSGD+: LOW-RANK GRADIENT COMPRESSION FOR DISTRIBUTED OPTIMIZATION WITH CONVERGENCE GUARANTEES

Low-rank gradient compression methods, such as PowerSGD, have gained attention in communication-efficient distributed optimization. However, the convergence guarantees of PowerSGD remain unclear, particularly in stochastic settings. In this paper, we show that PowerSGD does not always converge to th…