Pair2Scene: Learning Local Object Relations for Procedural Scene Generation

Abstract

Generating high-fidelity 3D indoor scenes remains a significant challenge due to data scarcity and the complexity of modeling intricate spatial relations. Current methods often struggle to scale beyond training distribution to dense scenes or rely on Large Language Models (LLMs) that lack the ability for precise spatial reasoning. Building on top of the observation that object placement relies mainly on local dependencies instead of information-redundant global distributions, in this paper, we propose **Pair2Scene**, a novel procedural generation framework, for scene generation based on a set of *learned procedural rules*. These rules mainly capture two types of inter-object relations, namely *support relations* that follow physical hierarchies, and *functional relations* that reflect semantic links. We model these rules through a network, which estimates spatial position distributions of dependent objects conditioned on position and geometry of the anchor ones. Accordingly, we curate a dataset **3D-Pairs** from existing scene data to train the model. During inference, our framework can generate scenes by recursively applying our model. Extensive experiments demonstrate that our framework outperforms existing methods in generating complex environments that go beyond training data while maintaining physical and semantic plausibility.