VINS-MLD2: Monocular Visual-Inertial SLAM With Multi-level Detector and Descriptor

Abstract

The performance of a vision simultaneous localization and mapping (SLAM) system based on hand-crafted features degrades significantly in harsh environments due to unstable feature tracking. With the breakthrough of convolutional neural networks in deep feature extraction tasks, many researchers have tried to incorporate them into SLAM systems. However, it’s challenging to guarantee the real-time performance of the entire SLAM system, and the erroneous usage scenarios limit the superior performance of deep feature extraction methods. To overcome these problems, we propose a visual-inertial SLAM system with multi-level detector and descriptor, called VINS-MLD2. In our framework, we first design an efficient deep feature extraction network that has the same performance as R2D2 by concatenating multi-level features, but runs 3 times faster under the image resolution commonly used in SLAM. Then, based on the camera baseline, we introduce the Matching Fusion, a matching method that fuses deep descriptor matching and optical flow matching results to improve matching accuracy for both short and wide baselines. In addition, an adaptive matching strategy is proposed to balance the running time and accuracy by adaptively adjusting the matching method. Experimental results in unmanned aerial vehicle (UAV) deployments and real-world environments demonstrate that the proposed method tracks features more stably and accurately. The code is public at https://github.com/dongdong-cai/VINS-MLD2.