Yoshihiro Yamada

CAT: Circular-Convolutional Attention for Sub-Quadratic Transformers

Transformers have driven remarkable breakthroughs in natural language processing and computer vision, yet their standard attention mechanism still imposes $O(N^2)$ complexity, hindering scalability to longer sequences. We introduce Circular-convolutional ATtention (CAT), a Fourier-based approach tha…