EventMASK: A Frame-Free Rapid Human Instance Segmentation With Event Camera Through Constrained Mask Propagation

Abstract

Human Instance Segmentation (HIS) is essential in robotics for applications such as autonomous driving and human-robot interaction, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">etc</i> . Existing HIS solutions using conventional cameras are computationally expensive and slow. Benefits such as sparsity, high temporal resolution, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">etc.</i> make the event camera a promising alternative. HIS with an event camera is not actively explored, though. Thus, we introduce EventMASK, a novel HIS solution that makes use of an event camera. EventMASK has been meticulously designed to process sparse raw events asynchronously, enabling low-latency processing. EventMASK employs simple statistical and probabilistic non-deep learning techniques for computational efficiency and adopts mask propagation for real-time performance. To curtail error accumulation, we present an innovative constrained likelihood-based mask updation method. EventMASK's semi-supervised approach circumvents the need for event-level instance labeling. The comprehensive analysis demonstrates EventMASK's robustness in a wide spectrum of scenarios, offering a low-cost and low-latent HIS solution for resource-constrained robotics.