PPR: Physically Plausible Reconstruction from Monocular Videos
Gengshan Yang, Shuo Yang, John Z. Zhang, Zachary Manchester, Deva Ramanan
Abstract
Given monocular videos, we build 3D models of articulated objects and environments whose 3D configurations satisfy dynamics and contact constraints. At its core, our method leverages differentiable physics simulation to aid visual reconstructions. We couple differentiable physics simulation with differentiable rendering via coordinate descent, which enables end-to-end optimization of, not only 3D reconstructions, but also physical system parameters from videos. We demonstrate the effectiveness of physics-informed reconstruction on monocular videos of quadruped animals and humans. It reduces reconstruction artifacts (e.g., scale ambiguity, unbalanced poses, and foot swapping) that are challenging to address by visual cues alone, and produces better foot contact estimation.
BibTeX
@inproceedings{iccv2023_pprphysicallypla,
title = {PPR: Physically Plausible Reconstruction from Monocular Videos},
author = {Gengshan Yang and Shuo Yang and John Z. Zhang and Zachary Manchester and Deva Ramanan},
booktitle = {ICCV 2023},
year = {2023}
}