Event-Based 6-DoF Object Tracking With Distance Field Reaching 130Hz

Abstract

Event cameras report asynchronous, per-pixel brightness changes with microsecond latency, making them attractive for high-speed 6-DoF object tracking. We present EDFT, a real-time 6-DoF object tracker that uses only a monocular event camera and a 3D model. At the core is an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">event-based distance field</i> (EDF) that endows sparse events with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">two-sided</i> spatial gradients around object edges—overcoming the one-sided behavior of Time Surface maps—and a 3D–2D registration that aligns a sparse, model-derived point set to the EDF. We evaluate on the E-POSE benchmark and on the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5events16</i> dataset we introduce with this article. On 5events16 (UR10 arm providing ground truth poses, five objects, ten sequences plus 6 handheld sequences), EDFT tracks accurately all sequences successfully contrary to three previous state-of-the-art methods. On E-POSE where objects occupy only a small fraction of the field of view and the camera moves rapidly, EDFT still maintains start-to-finish tracking.