2021
Redesigning the Transformer Architecture with Insights from Multi-particle Dynamical Systems
NeurIPS 2021spotlight
The Transformer and its variants have been proven to be efficient sequence learners in many different domains. Despite their staggering success, a critical issue has been the enormous number of parameters that must be trained (ranging from $10^7$ to $10^{11}$) along with the quadratic complexity of…