Kaiyuan Liao

A Global Past-Future Early Exit Method for Accelerating Inference of Pre-trained Language Models

Early exit mechanism aims to accelerate the inference speed of large-scale pre-trained language models. The essential idea is to exit early without passing through all the inference layers at the inference stage. To make accurate predictions for downstream tasks, the hierarchical linguistic informat…