Human Motion Generation via Conditioned GMVAE with TUNet

Abstract

In recent years, Variational Autoencoders (VAEs) have been proposed for motion synthesis to model action-label-conditioned human motion. However, these approaches only use Gaussian distribution as a prior assumption, this hard constraint might be too restrictive for the latent space and hurt the performance of the model. To address the issues, we model the latent space as a Gaussian mixture distribution and derive a new evidence lower bound (ELBO). Furthermore, to enhance the expressiveness of the model, we introduce Fisher discriminant as a regularization. We develop the attention mechanism and enable the Transformer-based U-Net to generate motions that correspond to semantic information only using action labels. The proposed CGMVAE-TU model has been evaluated on various datasets, and it surpasses the SOTA on almost all metrics. The generated human motions are realistic and natural.