John Guibas

Efficient Token Mixing for Transformers via Adaptive Fourier Neural Operators

Vision transformers have delivered tremendous success in representation learning. This is primarily due to effective token mixing through self attention. However, this scales quadratically with the number of pixels, which becomes infeasible for high-resolution inputs. To cope with this challenge, we…