GLST-GCN: Global-Local Spatio-Temporal Graph Convolutional Network for Skeleton-based Hand Motion Prediction

Abstract

This paper introduces a new task: skeleton-based hand motion sequence prediction, which can be applied to VR/AR systems and human-computer interaction systems to enhance user experience. To tackle this task, we performed a comprehensive analysis of hand movement patterns and propose a Global-Local Spatio-Temporal Graph Convolutional Network (GLST-GCN). Based on the local and global correlation characteristics of hand motions, the GLS block of GLST-GCN is proposed to extract spatial features through local branches and global modules. Additionally, we considered the variation in hand movement speeds, which leads to inconsistencies in temporal features across different time scales, and thus proposed the MGLT block to model both global and local temporal dependencies across multiple scales. We also conduct extensive experiments based on the remaked Bighand2.2M and FPHA datasets, and numerical results show that our proposed method offers state-of-the-art performance.