Kanghwan Jang

Composite Slice Transformer: An Efficient Transformer with Composition of Multi-Scale Multi-Range Attentions

Since the introduction of Transformers, researchers have tackled the notoriously expensive quadratic complexity problem. While significant computational efficiency improvements have been achieved, they come at the cost of reduced accuracy trade-offs. In this paper, we propose Composite Slice Transf…