DSSM: Dual State Space Model For Human Motions Generation

Abstract

Text-driven human motion generation has attracted considerable critical attention in recent years. The task requires generating movements that are diverse, natural, and comfortable in accordance with the text description. However, while generating the human motion, there is a significant gap in the amount of feature information contained in word-based text description modality and joint-based human motion modality. This results in an extreme imbalance of feature information in the latent space across the modalities, which seriously affects the effect of feature fusion. To alleviate the imbalance between modalities, we propose the Dual State Space Model (DSSM), which reconstruction the fused feature from coarse-to-fine. The DSSM contains two unit structures: the Masked State Space Model (MSSM) and the Hierarchical State Space Block (HSSB). At the same time, in order to make better use of timing information and reduce the computational complexity of the model, the DSSM is also the first method to introduce the state space model (SSM) into the text-driven motion sequence generation. We evaluated the DSSM on the HumanML3D and KITML benchmark datasets, and the experimental results show that our approach achieves state-of-the-art performance. The source code is available on GitHub at https://github.com/yimingliu123/DSSM.