High-Fidelity Single-View Reconstruction of Indoor Scenes using 3D Shape Prior Template and Pixel-Aligned Deformation

Abstract

This paper presents a novel pipeline for estimating room layouts and reconstructing the 3D shapes of indoor objects. This task remains challenging due to occlusions of indoor scenes, which lead to incomplete shape and poor geometric quality manifested as non-smooth meshes. Our key insight is that occlusions of indoor objects inherently lead to insufficient information in images. Pixel-level features alone are inadequate to recover the complete structure of objects; Therefore, additional prior information is required to supplement the missing occluded details. To address this, we propose a two-stage training strategy. First, a VQ-VAE encodes 3D shapes into a latent space, with a decoder leveraging priors to predict occluded regions. Second, pixel-aligned features are used for deformation, ensuring consistency between the reconstructed shape and the image. Quantitative and qualitative evaluations on the 3D-FRONT and SUNRGB-D datasets demonstrate that our approach surpasses state-of-the-art methods in reconstructing more complete geometric topologies and smoother meshes.