Feature-aided Bundle Adjustment Learning Framework for Self-supervised Monocular Visual Odometry

Abstract

Bundle adjustment refines scene geometry and relative camera poses simultaneously via reprojection error, computed by a set of images from different viewpoints, which is the gold standard for visual odometry. However, deep learning methods have not been well exploited within this area of study. This paper introduces a self-supervised learning framework for monocular visual odometry, inside which depth maps, relative camera poses, and dense feature maps (with the same resolution as images) are estimated and used for photometric, geometric, and feature-metric losses in bundle adjustment. In this manner, we consider that the learning of neural networks can be geometrically constrained by multi-view geometry. Furthermore, bundle adjustment is only required during the training time, allowing the networks to benefit from bundle adjustment without any additional computation burden during the inference time. To stabilize the training process, we apply a two-stage strategy that yields promising results. Finally, we carefully select the neural network architectures to ensure efficiency, and experimental results demonstrate the success of our proposed approach in terms of visual odometry accuracy and high speed.