Composition of Motion from Video Animation Through Learning Local Transformations

Abstract

In this work, we solve the problem of motion representation in videos, according to local transformations applied to specific keypoints extracted from static the images. First, we compute the co-ordinates of the keypoints of the body or face through a pre-trained model, and then we introduce a convolutional neural network to estimate a dense motion field through optical flow. Next, we train a generative adversarial network that exploits the previous information to generate new images that resemble as much as possible the target frames. To reduce trembling and extract smooth movements, our model incorporates a low-pass spatio-temporal Gaussian filter. Results indicate that our method provides high performance and the movement of objects is accurate and robust.