2023
Pyramid Dynamic Inference: Encouraging Faster Inference Via Early Exit Boosting
ICASSP 2023accepted
Transformer-based models demonstrate state of the art results on several natural language understanding tasks. However, their deployment comes at the cost of increased footprint and inference latency, limiting their adoption to real-time applications. Early exit strategies are designed to speed-up t…