Xueqing Huang

Empowering Adaptive Early-Exit Inference with Latency Awareness

With the capability of trading accuracy for latency on-the-fly, the technique of adaptive early-exit inference has emerged as a promising line of research to accelerate the deep learning inference. However, studies in this line of research commonly use a group of thresholds to control the accuracy-l…