SPFEMD: Super-pixel Based Finger Earth Mover's Distance for Hand Gesture Recognition

Abstract

In this paper, we propose super-pixel based finger earth mover's distance (SPFEMD) for hand gesture recognition. For finger representation, we design SPFEMD for similarity measurement between fingers and hand gestures, and use it as the distance metric for hand gesture recognition. First, we extract hands without any user interaction using both color and depth from Kinect camera. Then, we obtain the seed for hand segmentation on the depth map and segment hand on the color image using the seed. Since fingers contain distinct features for hand gesture recognition, we decompose the hand segment into palm and fingers based on morphological operation. Finally, we perform hand gesture recognition from fingers based on SPFEMD. Experiments on publicly available and our own data sets show the superiority of the proposed method over state-of-the-arts in terms of accuracy and confusion matrices.