A Finite-Particle Convergence Rate for Stein Variational Gradient Descent

Jiaxin Shi; Lester Mackey

NeurIPS 2023poster29 citations

A Finite-Particle Convergence Rate for Stein Variational Gradient Descent

Abstract

We provide the first finite-particle convergence rate for Stein variational gradient descent (SVGD), a popular algorithm for approximating a probability distribution with a collection of particles. Specifically, whenever the target distribution is sub-Gaussian with a Lipschitz score, SVGD with $n$ particles and an appropriate step size sequence drives the kernel Stein discrepancy to zero at an order ${1/}{\sqrt{\log\log n}}$ rate. We suspect that the dependence on $n$ can be improved, and we hope that our explicit, non-asymptotic proof strategy will serve as a template for future refinements.

Stein Variational Gradient DescentSVGDvariational inferencesamplingoptimizationStein's method

BibTeX

@inproceedings{
shi2023a,
title={A Finite-Particle Convergence Rate for Stein Variational Gradient Descent},
author={Jiaxin Shi and Lester Mackey},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
year={2023},
url={https://openreview.net/forum?id=0eRDQQK2TW}
}