Sampling Nonsmooth Log-Concave Densities: A Comparative Study of Primal-Dual Based Proposal Distributions

Abstract

Sampling from a distribution on the real d-space, whose density is nonsmooth and log-concave, is a computational issue that often arises in Machine Learning and Statistics. Langevin-based Hastings-Metropolis methods were proposed: they extend the Unadjusted Langevin Algorithm by using proximal methods to define a smoothed version of the density of interest. We consider the case when these extensions do not apply: the involved proximal operators do not have closed-form expressions and the density is defined on a subset of the real d-space. We derive new Gaussian proposal mechanisms in a Metropolis Adjusted Langevin Algorithm, which use first-order information about the density function. We numerically compare these strategies and discuss the benefits of a change of geometry. The gain in using partial updates of the parameter instead of global updates is also illustrated.