Disentangled 3D Scene Generation with Layout Learning
Dave Epstein, Ben Poole, Ben Mildenhall, Alexei A Efros, Aleksander Holynski
Abstract
We introduce a method to generate 3D scenes that are disentangled into their component objects. This disentanglement is unsupervised, relying only on the knowledge of a large pretrained text-to-image model. Our key insight is that objects can be discovered by finding parts of a 3D scene that, when rearranged spatially, still produce valid configurations of the same scene. Concretely, our method jointly optimizes multiple NeRFs---each representing its own object---along with a *set of layouts* that composite these objects into scenes. We then encourage these composited scenes to be in-distribution according to the image generator. We show that despite its simplicity, our approach successfully generates 3D scenes decomposed into individual objects, enabling new capabilities in text-to-3D content creation.
BibTeX
@inproceedings{
epstein2024disentangled,
title={Disentangled 3D Scene Generation with Layout Learning},
author={Dave Epstein and Ben Poole and Ben Mildenhall and Alexei A Efros and Aleksander Holynski},
booktitle={Forty-first International Conference on Machine Learning},
year={2024},
url={https://openreview.net/forum?id=Lgh8bhWpVC}
}