Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild

Dominik Kulon; Riza Alp Guler; Iasonas Kokkinos; Michael M. Bronstein; Stefanos Zafeiriou

CVPR 2020oral247 citations

Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild

Dominik Kulon, Riza Alp Guler, Iasonas Kokkinos, Michael M. Bronstein, Stefanos Zafeiriou

Abstract

We introduce a simple and effective network architecture for monocular 3D hand pose estimation consisting of an image encoder followed by a mesh convolutional decoder that is trained through a direct 3D hand mesh reconstruction loss. We train our network by gathering a large-scale dataset of hand action in YouTube videos and use it as a source of weak supervision. Our weakly-supervised mesh convolutions-based system largely outperforms state-of-the-art methods, even halving the errors on the in the wild benchmark. The dataset and additional resources are available at https://arielai.com/mesh_hands.

BibTeX

@inproceedings{cvpr2020_weaklysupervised,
  title = {Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild},
  author = {Dominik Kulon and Riza Alp Guler and Iasonas Kokkinos and Michael M. Bronstein and Stefanos Zafeiriou},
  booktitle = {CVPR 2020},
  year = {2020}
}