Konstantinos Christopher Tsiolis

From Information to Generative Exponent: Learning Rate Induces Phase Transitions in SGD

To understand feature learning dynamics in neural networks, recent theoretical works have focused on gradient-based learning of Gaussian single-index models, where the label is a nonlinear function of a latent one-dimensional projection of the input. While the sample complexity of online SGD is dete…