Grasp-and-Classify Robotic Sorting With Grasping Rectangle Correction and Weighted Nearest-Neighbor Relation Network

Abstract

Robotic sorting in cluttered environments still faces significant challenges, especially with resource-constrained hardware. Traditional detect-and-grasp workflows usually require extensive image collection and annotation for model training, which can become impractical when the categories of the sorted objects frequently change. To overcome this issue, this article proposes a grasp-and-classify robotic sorting method with deep learning-based object grasping and classification algorithms which can be deployed on resource-constrained hardware platforms. To do this, a Grasping Rectangle Correction (GRC) algorithm is incorporated to adjust the grasping poses generated from the Generative Residual Convolutional Neural Network (GR-ConvNetv2). Then, an efficient Weighted Nearest-Neighbor Relation Network (WNNRNet) is developed for few-shot object classification. This model unifies Deep Nearest Neighbor Neural Network (DN4) and Relation network to reduce overfitting through feature sharing, and the joint training with a weighted multi-task loss function can enhance the generalization capability of few-shot classification. Simulation tests have been carried out to validate the GRC and WNNRNet algorithms with Cornell, Jacquard, and MiniImageNet datasets. Finally, a robotic sorting system with a UR10 robot and a Kinect camera has been built to perform real-world sorting tests to demonstrate the effectiveness of the proposed method. Benefited from the efficient correction of the grasping pose with the GRC algorithm and the fact that the WNNRNet requires limited samples for training, the proposed method can be deployed on a consumer-level laptop for sorting stacked objects in scenarios with varying categories.