Katherine A Heller — accepted AI-conference papers

2025

AfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering Benchmark Dataset

Charles Nimo, Tobi Olatunji, Abraham Toluwase Owodunni, Tassallah Abdullahi, Emmanuel Ayodele, Mardhiyah Sanni +20

ACL 2025long

Recent advancements in large language model (LLM) performance on medical multiplechoice question (MCQ) benchmarks have stimulated interest from healthcare providers and patients globally. Particularly in low-andmiddle-income countries (LMICs) facing acute physician shortages and lack of specialists,…

Cited by 0 Source ↗PDF ↗Scholar ↗

2025

Position: Stop treating `AGI' as the north-star goal of AI research

Borhane Blili-Hamelin, Christopher Graziul, Leif Hancox-Li, Hananel Hazan, El-Mahdi El-Mhamdi, Avijit Ghosh +10

ICML 2025poster

The AI research community plays a vital role in shaping the scientific, engineering, and societal goals of AI research. In this position paper, we argue that focusing on the highly contested topic of 'artificial general intelligence' ('AGI') undermines our ability to choose effective goals. We ident…

Cited by 0 Source ↗PDF ↗Scholar ↗

2025

Speech Recognition with LLMs Adapted to Disordered Speech Using Reinforcement Learning

Chirag Nagpal, Subhashini Venugopalan, Jimmy Tobin, Marilyn A. Ladewig, Katherine A. Heller, Katrin Tomanek

ICASSP 2025accepted

We introduce a large language model (LLM) capable of processing speech inputs and show that tuning it further with reinforcement learning on human preference (RLHF) enables it to adapt better to disordered speech than traditional fine-tuning. Our method replaces low-frequency text tokens in an LLM’s…

Cited by 0 Source ↗Scholar ↗

2025

Understanding challenges to the interpretation of disaggregated evaluations of algorithmic fairness

Stephen R Pfohl, Natalie Harris, Chirag Nagpal, David Madras, Vishwali Mhasawade, Olawale Elijah Salaudeen +9

NeurIPS 2025poster

Disaggregated evaluation across subgroups is critical for assessing the fairness of machine learning models, but its uncritical use can mislead practitioners. We show that equal performance across subgroups is an unreliable measure of fairness when data are representative of the relevant populations…

Cited by 0 Source ↗code ↗Scholar ↗

2025

What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models

Ahmed Imtiaz Humayun, Ibtihel Amara, Cristina Nader Vasconcelos, Deepak Ramachandran, Candice Schumann, Junfeng He +4

ICLR 2025poster

Deep Generative Models are frequently used to learn continuous representations of complex data distributions by training on a finite number of samples. For any generative model, including pre-trained foundation models with Diffusion or Transformer architectures, generation performance can significan…

Cited by 2 Source ↗PDF ↗code ↗Scholar ↗

2024

Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering

Han Zhou, Xingchen Wan, Lev Proleev, Diana Mincu, Jilin Chen, Katherine A Heller +1

ICLR 2024poster

Prompting and in-context learning (ICL) have become efficient learning paradigms for large language models (LLMs). However, LLMs suffer from prompt brittleness and various bias factors in the prompt, including but not limited to the formatting, the choice verbalizers, and the ICL examples. To addres…

Cited by 51 Source ↗PDF ↗Scholar ↗

2023

Participatory Personalization in Classification

Hailey Joren, Chirag Nagpal, Katherine A Heller, Berk Ustun

NeurIPS 2023spotlight

Machine learning models are often personalized based on information that is protected, sensitive, self-reported, or costly to acquire. These models use information about people, but do not facilitate nor inform their *consent*. Individuals cannot opt out of reporting information that a model needs t…

Cited by 6 Source ↗PDF ↗Scholar ↗

2022

Diagnosing failures of fairness transfer across distribution shift in real-world medical settings

Jessica Schrouff, Natalie Harris, Oluwasanmi O Koyejo, Ibrahim Alabdulmohsin, Eva Schnider, Krista Opsahl-Ong +11

NeurIPS 2022accept

Diagnosing and mitigating changes in model fairness under distribution shift is an important component of the safe deployment of machine learning in healthcare settings. Importantly, the success of any mitigation strategy strongly depends on the \textit{structure} of the shift. Despite this, there h…

Cited by 74 Source ↗PDF ↗Scholar ↗

2021

Variational refinement for importance sampling using the forward Kullback-Leibler divergence

Ghassen Jerfel, Serena Wang, Clara Wong-Fannjiang, Katherine A. Heller, Yian Ma, Michael I. Jordan

UAI 2021poster

Variational Inference (VI) is a popular alternative to asymptotically exact sampling in Bayesian inference. Its main workhorse is optimization over a reverse Kullback-Leibler divergence (RKL), which typically underestimates the tail of the posterior leading to miscalibration and potential degeneracy…

Cited by 44 Source ↗PDF ↗Scholar ↗

2019

Reconciling meta-learning and continual learning with online mixtures of tasks

Ghassen Jerfel, Erin Grant, Tom Griffiths, Katherine A. Heller

NeurIPS 2019spotlight

Learning-to-learn or meta-learning leverages data-driven inductive bias to increase the efficiency of learning on a novel task. This approach encounters difficulty when transfer is not advantageous, for instance, when tasks are considerably dissimilar or change over time. We use the connection betwe…

Cited by 142 Source ↗PDF ↗Scholar ↗

2017

An inner-loop free solution to inverse problems using deep neural networks

Kai Fan, Qi Wei, Lawrence Carin, Katherine A. Heller

NeurIPS 2017poster

We propose a new method that uses deep learning techniques to accelerate the popular alternating direction method of multipliers (ADMM) solution for inverse problems. The ADMM updates consist of a proximity operator, a least squares regression that includes a big matrix inversion, and an explicit so…

Cited by 27 Source ↗PDF ↗Scholar ↗

2015

Fast Second Order Stochastic Backpropagation for Variational Inference

Kai Fan, Ziteng Wang, Jeff Beck, James Kwok, Katherine A. Heller

NeurIPS 2015poster

We propose a second-order (Hessian or Hessian-free) based optimization method for variational inference inspired by Gaussian backpropagation, and argue that quasi-Newton optimization can be developed as well. This is accomplished by generalizing the gradient computation in stochastic backpropagatio…

Cited by 52 Source ↗PDF ↗Scholar ↗

2015

Parallelizing MCMC with Random Partition Trees

Xiangyu Wang, Fangjian Guo, Katherine A. Heller, David B Dunson

NeurIPS 2015poster

The modern scale of data has brought new challenges to Bayesian inference. In particular, conventional MCMC algorithms are computationally very expensive for large data sets. A promising approach to solve this problem is embarrassingly parallel MCMC (EP-MCMC), which first partitions the data into m…

Cited by 79 Source ↗PDF ↗code ↗Scholar ↗