Enhancing 3D Scene Graphs with Real-Time Room Classification

Abstract

In recent years, 3D scene graphs have become a critical tool in robotics and computer vision for enabling systems to understand both the geometric and semantic aspects of their surroundings. These data structures represent spatial and semantic relationships between objects in a three-dimensional environment, supporting tasks like navigation, object manipulation, and scene understanding. This paper presents a real-time pipeline for 3D scene graph generation that offers flexibility in image segmentation techniques while incorporating room classification that is based on a Random Forest model. Our work enables robots to dynamically update their understanding of complex and large-scale environments in real-time. We evaluate our approach systematically on a dataset and in a real-life experiment. The results demonstrate the capability of running our solution at over 10 Hz on an Nvidia Jetson AGX Orin SoC while also scaling favorably in larger environments. Our proposed room classification approach predicts classes with an average accuracy of 80%.