NeurIPS 2020spotlight223 citations
Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses
Raef Bassily, Vitaly Feldman, Cristóbal Guzmán, Kunal Talwar
Abstract
Uniform stability is a notion of algorithmic stability that bounds the worst case change in the model output by the algorithm when a single data point in the dataset is replaced. An influential work of Hardt et al. [2016] provides strong upper bounds on the uniform stability of the stochastic gradient descent (SGD) algorithm on sufficiently smooth convex losses. These results led to important progress in understanding of the generalization properties of SGD and several applications to differentially private convex optimization for smooth losses.
BibTeX
@inproceedings{NEURIPS2020_2e2c4bf7,
author = {Bassily, Raef and Feldman, Vitaly and Guzm\'{a}n, Crist\'{o}bal and Talwar, Kunal},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Larochelle and M. Ranzato and R. Hadsell and M.F. Balcan and H. Lin},
pages = {4381--4391},
publisher = {Curran Associates, Inc.},
title = {Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses},
url = {https://proceedings.neurips.cc/paper_files/paper/2020/file/2e2c4bf7ceaa4712a72dd5ee136dc9a8-Paper.pdf},
volume = {33},
year = {2020}
}