DLODepth: Real-Time Depth Recovery for 3D Reflective Deformable Linear Object

Abstract

An end-to-end monocular 3D recovery framework for Deformable Linear Object (DLO) is proposed in this paper. The fragmented and unreliable 3D point clouds caused by the thin profile and reflective surfaces of DLOs when captured with an RGB-D camera have been a critical challenge in 3D DLO perception. Conventional algorithms circumvent these issues by relying on simplified background environments or expensive multi-sensor systems, yet such constraints severely limit their practical downstream applications. Taking raw RGB-D data as input and recovered depth data as output, this paper proposes a mixed body of multiple branches, including RGB segmentation, relative depth estimation, relative-to-metric scaling transformation, and a recovery fusion module. Our framework achieves state-of-the-art performance in recovering DLO from highly noisy inputs, recovering 93.8% (median) of the target point cloud within a 5cm error band, with a mean distance error of 4.3cm. This represents a significant improvement over the raw depth data whose mean distance error is greater than 30cm. Moreover, the system operates in a real-time manner, enabling its seamless integration to existing DLO manipulation pipelines as a pre-processing module. A real-world robot DLO grasping demonstration, an open-sourcing implementation of the proposed algorithm, a GUI-based data collection tool, and a ready-to-use dataset have also been provided for the benefit of the community.