LBSNet: Lightweight Joint Boundary Detection and Semantic Segmentation for Transparent and Reflective Objects

Abstract

Accurate visual detection of transparent and reflective objects remains a challenging issue for mobile manipulators. For the most common depth cameras and LiDAR sensors, the distinctive optical attributes inherent in both transparent and reflective objects pose a significant challenge. To address this problem, this study proposes a lightweight joint boundary detection and semantic segmentation network named LBSNet. LBSNet aims to enhance the perception of transparent and reflective objects in complex and dynamic environments, using RGB images only. It leverages the synergy between boundary detection and semantic segmentation through feature fusion and a multitask learning mechanism. The encoder consists of two paths: one captures category-aware semantic information, while the other discerns boundaries from multi-scale features. The gated channel adaptive (GCA) module enhances boundary features by learning channel parameters. The dynamic adaptive feature fusion (DAFF) module dynamically adjusts semantic and boundary information through cross-feature fusion. These methods effectively capture the distinctive characteristics of transparent and reflective objects, such as light refraction, boundary blurring and low contrast. Experimental results show that LBSNet achieves higher accuracy and faster processing speed on multiple public datasets compared with existing methods. Moreover, its lightweight design makes it suitable for resource-constrained mobile manipulators.