Alfredo Canziani — accepted AI-conference papers

2026

Anatomy of Massive Activations and Attention Sinks

Shangwen Sun, Alfredo Canziani, Yann LeCun, Jiachen Zhu

ICML 2026poster

We study two recurring phenomena in Transformer language models. First, \emph{massive activations}, where a small number of hidden channels attain extremely large values for a few tokens. Second, \emph{attention sinks}, where certain tokens attract a disproportionate share of attention across many h…

Cited by 0 Source ↗Scholar ↗

2019

Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic

Mikael Henaff, Alfredo Canziani, Yann LeCun

ICLR 2019poster

Learning a policy using only observational data is challenging because the distribution of states it induces at execution time may differ from the distribution observed during training. In this work, we propose to train a policy while explicitly penalizing the mismatch between these two distribution…

Cited by 159 Source ↗PDF ↗code ↗Scholar ↗