Efficient Posenet with Coarse to Fine Transformer

Abstract

In recent years, Transformers have been widely applied in human pose estimation by converting image features into token forms as inputs. However, redundant information in images burdens the network and can even negatively impact training as noise. Thus, we propose a coarse-to-fine Transformer called CFPose for efficient human pose estimation. We first extract visual features through a backbone network, then remove redundancy and coarsely crop the human figure via a coarse-grained decision network. Coarse-Grained token (CG token) and keypoint token are fed into a two-stage Transformer, where after the CG encoder, keypoint token sufficiently incorporate coarse features. Only keypoint token and Fine-Grained token (FG token) from a fine-grained decision network that further segments features are input to the FG encoder for training. Finally, keypoint token are mapped to 2D heatmaps for keypoint prediction. Impressively, CFPose reduces computational complexity by 43% while improving accuracy to 76.2% on COCO. It also achieves competitive results on MPII.