Will Sussman Grathwohl — accepted AI-conference papers

2024

A Fresh Take on Stale Embeddings: Improving Dense Retriever Training with Corrector Networks

Nicholas Monath, Will Sussman Grathwohl, Michael Boratko, Rob Fergus, Andrew McCallum, Manzil Zaheer

ICML 2024poster

In dense retrieval, deep encoders provide embeddings for both inputs and targets, and the softmax function is used to parameterize a distribution over a large number of candidate targets (e.g., textual passages for information retrieval). Significant challenges arise in training such encoders in the…

Cited by 0 Source ↗PDF ↗Scholar ↗

2023

DISCS: A Benchmark for Discrete Sampling

Katayoon Goshvadi, Haoran Sun, Xingchao Liu, Azade Nova, Ruqi Zhang, Will Sussman Grathwohl +2

NeurIPS 2023poster

Sampling in discrete spaces, with critical applications in simulation and optimization, has recently been boosted by significant advances in gradient-based approaches that exploit modern accelerators like GPUs. However, two key challenges are hindering further advancement in research on discrete sam…

Cited by 5 Source ↗PDF ↗code ↗Scholar ↗

2023

Denoising Diffusion Samplers

Francisco Vargas, Will Sussman Grathwohl, Arnaud Doucet

ICLR 2023poster

Denoising diffusion models are a popular class of generative models providing state-of-the-art results in many domains. One adds gradually noise to data using a diffusion to transform the data distribution into a Gaussian distribution. Samples from the generative model are then obtained by simulatin…

Cited by 84 Source ↗PDF ↗Scholar ↗

2023

Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC

Yilun Du, Conor Durkan, Robin Strudel, Joshua B. Tenenbaum, Sander Dieleman, Rob Fergus +3

ICML 2023poster

Since their introduction, diffusion models have quickly become the prevailing approach to generative modeling in many domains. They can be interpreted as learning the gradients of a time-varying sequence of log-probability density functions. This interpretation has motivated classifier-based and cla…

Cited by 162 Source ↗PDF ↗code ↗Scholar ↗

2022

Learning to Navigate Wikipedia by Taking Random Walks

Manzil Zaheer, Kenneth Marino, Will Sussman Grathwohl, John Schultz, Wendy Shang, Sheila Babayan +4

NeurIPS 2022accept

A fundamental ability of an intelligent web-based agent is seeking out and acquiring new information. Internet search engines reliably find the correct vicinity but the top results may be a few links away from the desired target. A complementary approach is navigation via hyperlinks, employing a pol…

Cited by 5 Source ↗PDF ↗Scholar ↗

2022

Score-Based Diffusion meets Annealed Importance Sampling

Arnaud Doucet, Will Sussman Grathwohl, Alexander G. D. G. Matthews, Heiko Strathmann

NeurIPS 2022accept

More than twenty years after its introduction, Annealed Importance Sampling (AIS) remains one of the most effective methods for marginal likelihood estimation. It relies on a sequence of distributions interpolating between a tractable initial distribution and the target distribution of interest whic…

Cited by 52 Source ↗PDF ↗Scholar ↗

2021

No MCMC for me: Amortized sampling for fast and stable training of energy-based models

Will Sussman Grathwohl, Jacob Jin Kelly, Milad Hashemi, Mohammad Norouzi, Kevin Swersky, David Duvenaud

ICLR 2021poster

Energy-Based Models (EBMs) present a flexible and appealing way to represent uncertainty. Despite recent advances, training EBMs on high-dimensional data remains a challenging problem as the state-of-the-art approaches are costly, unstable, and require considerable tuning and domain expertise to app…

Cited by 14 Source ↗PDF ↗code ↗Scholar ↗