Killian Bakong

Stab-SGD: Noise-Adaptivity in Smooth Optimization with Stability Ratios

In the context of smooth stochastic optimization with first order methods, we introduce the stability ratio of gradient estimates, as a measure of local relative noise level, from zero for pure noise to one for negligible noise. We show that a schedule-free variant (Stab-SGD) of stochastic gradient…