MEFusion: Memory-Efficient Data Fusion for Real-Time 3D Reconstruction On Resource-Constrained Devices

Abstract

Online semantic 3D modeling from streaming RGB-D data fundamentally requires consistent fusion of 2D segmentation. Popular approaches address segmentation inconsistencies through histogram-based label aggregation, where each 3D element (point/voxel) maintains the frequency of candidate labels, which introduces prohibitive memory and computational overhead for resource-constrained devices. In response to this challenge, we propose MEFusion, a memory-efficient probabilistic fusion framework to avoid element-wise histogram aggregation. Specifically, we propose an element-wise probability update algorithm based on Bayesian Estimation, where each voxel stores only one instance label and updates it based on a posterior probability to maintain segmentation consistency. Following 3D segmentation, we establish a segment-wise voting framework to aggregate the semantic labels from historical data, where co-segment voxels share the semantic voting histogram, for semantic consistency. Our experiments demonstrate that our method achieves a memory reduction of 77%(85%) and a speed improvement of 58%(6.12x) on the desktop (embedded) platform while maintaining comparable reconstruction accuracy to the state-of-the-art point-cloud-based method.