A Sim-to-Real Instance Segmentation Framework for Densely Stacked Cartons

Abstract

Robotic picking systems in automated logistics require accurate segmentation and localization of densely stacked cartons. However, the lack of comprehensive and diverse datasets for this task poses a significant challenge. Furthermore, existing instance segmentation methods struggle to meet the accuracy requirements due to the unique characteristics of cartons, including their large quantity, small spatial gaps, similar postures, and complex textures. To address these issues, this letter proposes a sim-to-real instance segmentation framework specifically designed for densely stacked cartons. Synthetic data of densely stacked cartons is generated using an automated simulation pipeline, resulting in the Densely Stacked Object Dataset (DSOD) with 40 000 data samples and 1 209 951 individually segmented cartons in various environmental settings. The instance segmentation network employs gated fully fusion and disentangled non-local modules to prioritize salient boundaries, improving segmentation accuracy. Experimental evaluation on both the DSOD and an existing real-world carton dataset demonstrates that our method outperforms state-of-the-art techniques, achieving an impressive performance of 85.2 on AP50 and 78.7 on AP75. These findings highlight the effectiveness of our sim-to-real framework for accurately segmenting and localizing densely stacked cartons.