2026
Mitigating Per-Sample Harm in Stochastic Optimization
ICML 2026poster
Modern optimizers combine gradients from the current mini-batch with historical optimization state, such as momentum or adaptive moments. While effective for stability, this interaction can produce update directions that increase the loss of individual samples in the current batch. We formalize this…