A Coarse-to-Fine Event-based Framework for Camera Pose Relocalization with Spatio-Temporal Retrieval and Refinement Network

Abstract

Most existing event-based camera pose relocalization (CPR) learning methods implicitly encode environmental information into network parameters to achieve end-to-end mapping from event stream to pose. However, these end-to-end CPR methods fail to utilize prior environmental information effectively. As the scale of the environment increases, the difficulty of this mapping relationship grows significantly, reducing the robustness of the end-to-end methods across different scenarios. To address the above issues, this paper proposes the first coarse-to-fine event-based CPR framework, which achieves a new paradigm from end-to-end pose regression network to a hierarchical approach. In the coarse localization stage, we effectively encode similarity features by incorporating the fine-grained temporal information, achieving accurate retrieval of nearby event stream. In the pose refinement stage, we present an Event Spatio-temporal Pose Refinement Network (ESPR-Net) based on the Recurrent Convolutional Neural Networks (RCNN) architecture, which is capable of learning more nu-anced spatio-temporal features to achieve accurate regression of the relative pose. Finally, we conducted a comprehensive comparison on the IJRR and M3ED dataset, achieving state-of-the-art (SOTA) performance on both. Notably, our method attains a significant 83 % performance improvement on the outdoor M3ED dataset.