2023
Composite Slice Transformer: An Efficient Transformer with Composition of Multi-Scale Multi-Range Attentions
ICLR 2023poster
Since the introduction of Transformers, researchers have tackled the notoriously expensive quadratic complexity problem. While significant computational efficiency improvements have been achieved, they come at the cost of reduced accuracy trade-offs. In this paper, we propose Composite Slice Transf…