Real-Time Glass Detection and Reprojection Using Sensor Fusion Onboard Aerial Robots

Abstract

Autonomous aerial robots are increasingly being deployed in real-world scenarios, where transparent obstacles present significant challenges to reliable navigation and mapping. These materials pose a unique problem for traditional perception systems because they lack discernible features and can cause conventional depth sensors to fail, leading to inaccurate maps and potential collisions. To ensure safe navigation, robots must be able to accurately detect and map these transparent obstacles. Existing methods often rely on large, expensive sensors or algorithms that impose high computational burdens, making them unsuitable for low Size, Weight, and Power (SWaP) robots. We present a resource-constrained sensing pipeline for detecting and mapping transparent planar obstacles onboard a sub-300g quadrotor. By exploiting Time-of-Flight (ToF) speckle morphology and sonar-gated fusion, our system identifies specular reflections and reprojects their depth into empty space regions in real-time, with safety margins analytically validated for indoor flight speeds. The entire pipeline operates onboard an embedded processor using approximately 20% of a single CPU core at 2 Hz. We validate our system through experiments in controlled and real-world environments, confirming its ability to accurately render transparent obstacles visible. To our knowledge, this is the first CPU-only, real-time demonstration of transparent plane reprojection on a sub-300g quadrotor.