Subject-Embedded Vision Transformer with Transfer Learning for Cross-Subject Dynamic Hand Gesture Recognition Using HD-sEMG

Abstract

Hand gesture recognition (HGR) is crucial in developing advanced prosthetics, neurorobotics, and human-robot interaction (HRI). Surface electromyography (sEMG) and high-density sEMG (HD-sEMG) have gained attention for their ability to capture the muscle activity underlying hand gestures. Although many models achieve high performance within the same subjects, generalizing across different subjects remains a significant challenge, limiting the practical application of these systems in real-world settings. Furthermore, most conventional approaches primarily focus on the steady phase of gestures, which slows down real-time prediction. To address these issues, we propose a cross-subject dynamic hand gesture recognition (DHGR) framework based on the Vision Transformer (ViT) architecture, referred to as ViT-DHGR. Our model focuses explicitly on the signal transient phase before gesture stabilization to reduce gesture prediction latency and counteract system control delays. By incorporating subject embeddings and transfer learning strategies, the proposed ViT-DHGR framework for 34 dynamic hand gestures achieved an accuracy of 76.44% for 10 subjects using only 1 repetition of gesture data, which improved to 85.03% with 2 repetitions. In addition, our proposed framework achieves over 16% higher average accuracy across test subjects using 1 repetition of data compared to training subject-specific models from scratch. This work demonstrates the potential of HD-sEMG for capturing dynamic hand gestures and highlights the benefits of cross-user knowledge transfer in reducing data requirements and enhancing practicality for robotic applications.