Jinfan Zhang

PCEE-BERT: Accelerating BERT Inference via Patient and Confident Early Exiting

BERT and other pretrained language models (PLMs) are ubiquitous in modern NLP. Even though PLMs are the state-of-the-art (SOTA) models for almost every NLP task (CITATION), the significant latency during inference prohibits wider industrial usage. In this work, we propose Patient and Confident Early…