HEAPGrasp: Hand-Eye Active Perception to Grasp Objects With Diverse Optical Properties

Abstract

Autonomous robotic handling requires accurate 3-D scene measurement followed by grasp planning. Conventional systems struggle with transparent or specular objects. Additionally, in hand-eye setups, moving through multiple viewpoints increases handling execution time. In this paper, we propose HEAPGrasp–Hand-Eye Active Perception to Grasp objects with diverse optical properties. To measure such objects, we focus on the ability to segment objects regardless of their optical properties in RGB images. We employ Shape from Silhouette based on the segmented images for 3-D measurement. To shorten the time required for multi-view capture with a hand-eye camera, we plan its trajectory using a cost function that balances 3-D measurement accuracy against its trajectory length. Real-robot experiments achieve a 96.0% grasp success rate on transparent, specular, and opaque objects, while reducing the hand-eye camera's trajectory length by 52% and handling execution time by 19% relative to a baseline that circles around the scene for 3-D measurement.