2026
Ensembling Pruned Attention Heads For Uncertainty-Aware Efficient Transformers
ICLR 2026poster
Uncertainty quantification (UQ) is essential for deploying deep neural networks in safety-critical settings. Although methods like Deep Ensembles achieve strong UQ performance, their high computational and memory costs hinder scalability to large models. We introduce Hydra Ensembles, an efficient tr…