Refining 3D Human Mesh via Model-Free Offsets Estimation

Abstract

3D human mesh reconstruction from a single RGB image is a challenging task. Existing methods either utilize parametric mesh models to restrain the 3D human structures or directly regress the 3D coordinates of the mesh vertices. The former ones, called model-based methods, usually fail to recover the high variance of human mesh due to limited capacity of parametric models, whereas the latter ones called model-free methods suffer from unrealistic human structures because of lack of 3D priors. To mitigate the drawbacks of them, we propose that the model-based reconstruction can serve as a good starting point for the model-free refinement, so that the 3D structure priors of the parametric model and the high representation capability of model-free methods can be both inherited. By building a model-free refinement head upon a pretrained model-based regressor, our method reduces the reconstruction errors of 3D human mesh on public datasets H36M and 3DPW, demonstrating the advantage of combining model-based and model-free methods together.