2022
Pyramid-BERT: Reducing Complexity via Successive Core-set based Token Selection
ACL 2022long
Transformer-based language models such as BERT (CITATION) have achieved the state-of-the-art performance on various NLP tasks, but are computationally prohibitive. A recent line of works use various heuristics to successively shorten sequence length while transforming tokens through encoders, in tas…