Ankit Vani*

SPARO: Selective Attention for Robust and Compositional Transformer Encodings for Vision

"Selective attention helps us focus on task-relevant aspects in the constant flood of our sensory input. This constraint in our perception allows us to robustly generalize under distractions and to new compositions of perceivable concepts. Transformers employ a similar notion of attention in their a…