Vassilis Apidopoulos

Taming Stochastic Gradient Descent: Almost Sure Convergence and Saddle-Point Avoidance under $(L_{0},L_{1})$-Smoothness

Many optimization problems in machine learning and data science—from deep neural networks to Bayesian inference and beyond—fall outside the standard Lipschitz smoothness framework that underpins the convergence theory of stochastic gradient descent (SGD). Motivated by this theory-practice disconnect…