Simulation-Ready Tree: High-Quality Dynamic Tree Reconstruction from a Single RGB-D Sensor

Abstract

Realistic animation of real trees is challenging due to the difficulty in accurately capturing and simulating their movements under varying environmental conditions. Most of real tree reconstruction methods focus on the static modeling of trees from RGB images or LiDAR point clouds. Rather than RGB images, RGB-D (RGB+Depth) sensors provide a low-cost solution for faithful reconstruction of dynamic tree models in 3D. However, it is difficult to capture and reconstruct a complete dynamic tree with complex branch structures using a single RGB-D sensor. In this paper, we propose Simulation-Ready Tree, a dynamic tree reconstruction framework that synthesizes simulation-ready trees by reconstructing 3D tree models and extracting material properties of tree branches from only a single RGB-D sensor. It starts by pre-scanning multi-view RGB-D images around an outdoor tree. For creating a complete static tree point cloud, we presented a coarse-to-fine registration method by considering the skeleton features of main branches of tree points from multi-views. Then, a static tree model is reconstructed from the registered point cloud using an improved space colonization algorithm. Subsequently, a DeT (deep RGB-D tracking) model is employed to track the movements of tree branches during pull-testing, and the material properties of the tree are approximated by Fourier transform and half-power bandwidth methods. Next, a simulation-ready tree is created by constructing its hierarchical structures with corresponding material properties. Finally, the modal analysis method of curved cantilever beams is applied to the simulation-ready tree for animating trees under static load. We demonstrate realistic animation results of our framework by comparing with the ground truth RGB-D data sequences for various tree species.