RealFusion: 360deg Reconstruction of Any Object From a Single Image
Luke Melas-Kyriazi, Iro Laina, Christian Rupprecht, Andrea Vedaldi
Abstract
We consider the problem of reconstructing a full 360deg photographic model of an object from a single image of it. We do so by fitting a neural radiance field to the image, but find this problem to be severely ill-posed. We thus take an off-the-self conditional image generator based on diffusion and engineer a prompt that encourages it to "dream up" novel views of the object. Using the recent DreamFusion method, we fuse the given input view, the conditional prior, and other regularizers in a final, consistent reconstruction. We demonstrate state-of-the-art reconstruction results on benchmark images when compared to prior methods for monocular 3D reconstruction of objects. Qualitatively, our reconstructions provide a faithful match of the input view and a plausible extrapolation of its appearance and 3D shape, including to the side of the object not visible in the image.
BibTeX
@inproceedings{cvpr2023_realfusion360deg,
title = {RealFusion: 360deg Reconstruction of Any Object From a Single Image},
author = {Luke Melas-Kyriazi and Iro Laina and Christian Rupprecht and Andrea Vedaldi},
booktitle = {CVPR 2023},
year = {2023}
}