Correlative3D: Inter-Object Correlation-Aware 3D Scene Understanding

Abstract

Holistic 3D scene understanding from a single image is challenging due to the information loss in 2D-to-3D reconstruction. Existing approaches either explore object properties independently or overlook their levels of correlation, leading to inaccurate estimations in complex scenes. To this end, we propose Correlative3D, a novel inter-object correlation-aware method for 3D scene understanding. Our method integrates a Scene Graph Attention Network to implicitly enhance features through a correlation-driven weighting strategy, selectively prioritizing relationships among objects. In addition, an auxiliary task pertaining to the relative arrangement of objects is formulated to impose explicit constraints. Furthermore, we introduce a novel 3DCIoU loss that sensitively responds to geometric variations in 3D bounding boxes. Extensive experiments demonstrate that our method produces more coherent scene layouts compared to existing methods.