HISR: Hybrid Implicit Surface Representation for Photorealistic 3D Human Reconstruction
Angtian Wang, Yuanlu Xu, Nikolaos Sarafianos, Robert Maier, Edmond Boyer, Alan Yuille, Tony Tung
Abstract
Neural reconstruction and rendering strategies have demonstrated state-of-the-art performances due, in part, to their ability to preserve high level shape details. Existing approaches, however, either represent objects as implicit surface functions or neural volumes and still struggle to recover shapes with heterogeneous materials, in particular human skin, hair or clothes. To this aim, we present a new hybrid implicit surface representation to model human shapes. This representation is composed of two surface layers that represent opaque and translucent regions on the clothed human body. We segment different regions automatically using visual cues and learn to reconstruct two signed distance functions (SDFs). We perform surface-based rendering on opaque regions (e.g., body, face, clothes) to preserve high-fidelity surface normals and volume rendering on translucent regions (e.g., hair). Experiments demonstrate that our approach obtains state-of-the-art results on 3D human reconstructions, and also shows competitive performances on other objects.
BibTeX
@article{Wang_Xu_Sarafianos_Maier_Boyer_Yuille_Tung_2024, title={HISR: Hybrid Implicit Surface Representation for Photorealistic 3D Human Reconstruction}, volume={38}, url={https://ojs.aaai.org/index.php/AAAI/article/view/28337}, DOI={10.1609/aaai.v38i6.28337}, abstractNote={Neural reconstruction and rendering strategies have demonstrated state-of-the-art performances due, in part, to their ability to preserve high level shape details. Existing approaches, however, either represent objects as implicit surface functions or neural volumes and still struggle to recover shapes with heterogeneous materials, in particular human skin, hair or clothes. To this aim, we present a new hybrid implicit surface representation to model human shapes. This representation is composed of two surface layers that represent opaque and translucent regions on the clothed human body. We segment different regions automatically using visual cues and learn to reconstruct two signed distance functions (SDFs). We perform surface-based rendering on opaque regions (e.g., body, face, clothes) to preserve high-fidelity surface normals and volume rendering on translucent regions (e.g., hair). Experiments demonstrate that our approach obtains state-of-the-art results on 3D human reconstructions, and also shows competitive performances on other objects.}, number={6}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Wang, Angtian and Xu, Yuanlu and Sarafianos, Nikolaos and Maier, Robert and Boyer, Edmond and Yuille, Alan and Tung, Tony}, year={2024}, month={Mar.}, pages={5298-5308} }