2022
Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers
ICML 2022spotlight
Vision transformers using self-attention or its proposed alternatives have demonstrated promising results in many image related tasks. However, the underpinning inductive bias of attention is not well understood. To address this issue, this paper analyzes attention through the lens of convex duality…