2024
LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding
ACL 2024long
We present LayerSkip, an end-to-end solution to speed-up inference of large language models (LLMs). First, during training we apply layer dropout, with low dropout rates for earlier layers and higher dropout rates for later layers, and an early exit loss where all transformer layers share the same e…