2022
Jump Self-attention: Capturing High-order Statistics in Transformers
NeurIPS 2022accept
The recent success of Transformer has benefited many real-world applications, with its capability of building long dependency through pairwise dot-products. However, the strong assumption that elements are directly attentive to each other limits the performance of tasks with high-order dependencies…