Self-Supervised Uncertainty-Guided Refinement for Robust Joint Optical Flow and Depth Estimation

Abstract

Jointly estimating the optical flow and depth tasks in real-world scenes presents considerable hurdles due to some phenomena, such as occlusion, ambiguous textures, and illumination variation. The lack of guidance from the labeled data makes these challenges harder to overcome. This paper presents a novel approach to learning the regions with high uncertainties in a self-supervised manner. Our method allows the network to learn these regions by leveraging its predictions’ confidence. The uncertainties are then utilized in the estimation and refinement processes for optical flow and depth. We also introduce a novel uncertainty-guided smoothness regularization technique that leverages the uncertainty map to increase the robustness through concentrating the smoothness on regions with low confidence scores. Our results in the KITTI real-world dataset demonstrate the effectiveness of our approach for enhancing the predictions, particularly in the challenging scenes, showcasing the potential of our approach for real-world applications.