SemP-NBV: Semantic-Aware Predictive Next-Best-View for Autonomous Plant 3D Reconstruction

Abstract

Three-dimensional (3D) Plant Phenotyping enables comprehensive trait analysis for evaluating plant growth in precision agriculture. Current phenotyping frameworks are low-throughput due to frequent manual intervention and inefficiencies in handling large volumes of repetitive data. Existing view planners for phenotyping are limited to static sampling methods or narrow focus on specific plant organs, restricting their utility in capturing the full complexity of plant structures. To address these limitations, we propose SemP-NBV, a novel semantic-aware predictive next-best-view approach for sample-efficient 3D plant phenotyping. Evaluated in a photorealistic simulator with 12 plant categories, SemPNBV achieves 15.3% more observed points than an even-space sampler at 8 images on average, while matching the reconstruction quality at 20 even-space images. We demonstrate that existing state-of-the-art predictive planners designed for artificial structures struggle with zero performance increase compared to even-space sampler for complex plant structures. Furthermore, our approach generates semantic information during reconstruction, reducing the need for post hoc semantic labeling, and streamlining the 3D phenotyping workflow. The project is available at https://github.com/ARLabXiang/SemPNBV.