Cheng Zhuang

Ganet: Unary Attention Reaches Pairwise Attention Via Implicit Group Clustering in Light-Weight CNNs

The attention mechanism has been widely explored to construct a long-range connection which is beyond the realm of convolutions. The two groups of attention, unary and pair-wise attention, seem like being incompatible as fire and water due to the completely different operations. In this paper, we pr…