Reliable Aerial Manipulation: Combining Visual Tracking with Range Sensing for Robust Grasping

Abstract

Reliable object localization is a critical challenge in drone-based aerial manipulation, particularly when objects are outside the camera's field of view. This paper presents a new approach to enhance drone reliability in aerial grasping tasks by integrating a 1D time-of-flight range sensor with a vision-based localization system. The range sensor, positioned beneath the drone, generates a detailed point cloud of the ground beneath the drone, allowing for precise object localization even when the drone hovers directly above the target. By combining visual tracking with realtime distance measurements, our system achieves a 96 % grasp success rate across 128 trials with diverse objects, representing a significant improvement over previous approaches. This method enables zero-shot grasping without prior knowledge of the objects, increasing versatility and robustness in complex, unstructured environments. The open-source software and hardware design of the platform provide a foundation for further research and development in the field of autonomous aerial manipulation.