Aaron J Havens — accepted AI-conference papers

2025

Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matching

Aaron J Havens, Benjamin Kurt Miller, Bing Yan, Carles Domingo-Enrich, Anuroop Sriram, Daniel S. Levine +7

ICML 2025poster

We introduce Adjoint Sampling, a highly scalable and efficient algorithm for learning diffusion processes that sample from unnormalized densities, or energy functions. It is the first on-policy approach that allows significantly more gradient updates than the number of energy evaluations and model s…

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗

2024

Fine-grained Local Sensitivity Analysis of Standard Dot-Product Self-Attention

Aaron J Havens, Alexandre Araujo, Huan Zhang, Bin Hu

ICML 2024poster

Self-attention has been widely used in various machine learning models, such as vision transformers. The standard dot-product self-attention is arguably the most popular structure, and there is a growing interest in understanding the mathematical properties of such attention mechanisms. This paper p…

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗

2024

Novel Quadratic Constraints for Extending LipSDP beyond Slope-Restricted Activations

Patricia Pauli, Aaron J Havens, Alexandre Araujo, Siddharth Garg, Farshad Khorrami, Frank Allgöwer +1

ICLR 2024poster

Recently, semidefinite programming (SDP) techniques have shown great promise in providing accurate Lipschitz bounds for neural networks. Specifically, the LipSDP approach (Fazlyab et al., 2019) has received much attention and provides the least conservative Lipschitz upper bounds that can be compute…

Cited by 6 Source ↗PDF ↗Scholar ↗

2024

On the Scalability and Memory Efficiency of Semidefinite Programs for Lipschitz Constant Estimation of Neural Networks

Zi Wang, Bin Hu, Aaron J Havens, Alexandre Araujo, Yang Zheng, Yudong Chen +1

ICLR 2024poster

Lipschitz constant estimation plays an important role in understanding generalization, robustness, and fairness in deep learning. Unlike naive bounds based on the network weight norm product, semidefinite programs (SDPs) have shown great promise in providing less conservative Lipschitz bounds with p…

Cited by 12 Source ↗PDF ↗code ↗Scholar ↗

2023

A Unified Algebraic Perspective on Lipschitz Neural Networks

Alexandre Araujo, Aaron J Havens, Blaise Delattre, Alexandre Allauzen, Bin Hu

ICLR 2023top-25%

Important research efforts have focused on the design and training of neural networks with a controlled Lipschitz constant. The goal is to increase and sometimes guarantee the robustness against adversarial attacks. Recent promising techniques draw inspirations from different backgrounds to design 1…

Cited by 60 Source ↗PDF ↗code ↗Scholar ↗

2023

Exploiting Connections between Lipschitz Structures for Certifiably Robust Deep Equilibrium Models

Aaron J Havens, Alexandre Araujo, Siddharth Garg, Farshad Khorrami, Bin Hu

NeurIPS 2023poster

Recently, deep equilibrium models (DEQs) have drawn increasing attention from the machine learning community. However, DEQs are much less understood in terms of certified robustness than their explicit network counterparts. In this paper, we advance the understanding of certified robustness of DEQs…

Cited by 10 Source ↗PDF ↗code ↗Scholar ↗