A data-set and a method for pointing direction estimation from depth images for human-robot interaction and VR applications

Abstract

3D pointing devices are indispensable in virtual reality (hereafter VR) and human-robot interaction scenarios. Existing devices are cumbersome or non-immersive or have a limited volume of operation. Hand gesture-based interfaces do not suffer from these problems and can be used for 3D pointing purposes. However, there is a lack of robust, accurate hand gesture-based pointing techniques which can be attributed to the non-existence of large and accurate data-set for the same. To overcome this barrier, we propose a data-set consisting of depth images with a large number (107000) of samples collected from 11 subjects, with accurate ground-truth and adequate variation in the orientation and distance of the hand w.r.t. the camera. We propose a 3D convolutional neural network based technique that works on the proposed data-set and achieves an accuracy of 94.49% for an angle error threshold of 10 degrees. The proposed data-set may be used for developing more accurate, robust, less computationally expensive methods.